| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Set default value for custom hooks directory
See merge request gitlab-org/gitaly!2184
|
|
Follow-ups for transactions
See merge request gitlab-org/gitaly!2188
|
|
When either transaction or Praefect sevrer metadata cannot be extracted
from the gRPC stream or environment, we currently return an
`os.ErrNotFound` error. Its error message is quite misleading in this
context, as it says "file does not exist" and there are no files in play
at all.
Let's improve this by using our own custom data types instead of
`os.ErrNotFound`.
|
|
Include Praefect usage in the usage ping
See merge request gitlab-org/gitaly!2180
|
|
|
|
Update gems to match those in the Rails repo
See merge request gitlab-org/gitaly!2144
|
|
Praefect: horizontal scaling of a single shard MVC
See merge request gitlab-org/gitaly!2189
|
|
Documenting usage of `distributed_reads_enabled` with
description of read distribution feature.
Part of https://gitlab.com/gitlab-org/gitaly/-/issues/2650
|
|
With introduction of praefect between GitLab and Gitaly it is
required to include info about Praefect into Usage Ping.
As Praefect is a transparent proxy between two listed above
the new parameter is added to the response and will be filled
by the Praefect after gathering info from Gitalies behind it.
Later it will be used to calculate the number of storages
to send statistics into Usage Ping.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2664
|
|
|
|
Do not relativize GitObjectDirectory and GitAlternateObjectDirectories
See merge request gitlab-org/gitaly!2176
|
|
|
|
Single-node transactions via pre-receive hook
See merge request gitlab-org/gitaly!2147
|
|
In order to allow for strong consistency for Git reference updates,
Gitaly nodes are expected to perform voting on the updates they intend
to perform. While the server side for this exists already in form of the
transaction manager and reference transaction service, the client-side
is not yet wired up.
In this first iteration, we implement the client side by reaching out to
the reference transaction service from the pre-receive hook. While this
won't yet catch all reference updates but only those performed as the
result of git-push(1), it serves as a first minimally viable
demonstration that a two-phase commit via Git hooks works.
One interesting bit is that starting the transaction is _not_ guarded by
a feature flag on the hook side, but only by presence of environment
variables that carry transaction information. As this environment
variable is only set in case Praefect has registered a transaction,
which in turn only happens if the "reference_transactions" feature flag
is set, we are still guarded.
Further note the hook is only implemented in the Go hook service, which
is in turn guarded by the "hooks_rpc" feature flag. As the work is
currently of experimental nature, it's deemed as a nice proof of concept
for the Go hooks RPC.
|
|
Now that we have transaction information passed through to us by
Praefect in case the reference transaction feature flag is turned on, we
need to pass it on to the Git hooks. Again, we follow the same approach
as chosen for the Praefect server connection information to make data
available via the environment of the Git hook.
|
|
When wiring up `PraefectFromEnv()` in the hooks service later, we're not
going to look up keys from the OS environment but instead from the array
of environment variables passed to us via gRPC. As a preparatory step,
this commit thus converts `PraefectFromEnv()` to get an array of
environment variables as parameters to make it more flexible.
|
|
Update activesupport to v6.0.3 and gitlab-labkit to v0.12.0
See merge request gitlab-org/gitaly!2178
|
|
We upgraded to Rails 6.0.3 in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31307, so update
the gitaly-ruby gems to match in order to save some space.
|
|
|
|
Praefect: horizontal scaling of a single shard MVC
Closes #2650
See merge request gitlab-org/gitaly!2162
|
|
Random distribution of reads to up to date gitaly nodes.
Up to date status of node verified based on the state of the
replication queue. Backoff strategy is to use primary node
if there are no up to date secondaries or an error occurred.
This feature can be enabled with 'distribution_of_reads_enabled'.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/2650
|
|
Fix re-electing demoted primaries
See merge request gitlab-org/gitaly!2181
|
|
Replace read-only feature flag with a config toggle
See merge request gitlab-org/gitaly!2173
|
|
Fixes a bug where the demoted status of a primary is not considered
when checking whether the new primary candidate is already elected
as the primary. This caused a problem where Praefect was not able to
re-elect a demoted primary as it was considered already elected.
|
|
Removes the feature-flag controlling read-only enforcement. The
feature flag only controlled enforcement in RPCs, making future
upgrades more difficult as some storages could already be in
read-only mode when the feature flag is removed. The feature flag
is replaced with a config toggle that also controls whether the
storages should be set read-only on a failover.
|
|
Add missing checksum to go.sum
See merge request gitlab-org/gitaly!2179
|
|
9738045a3 added a dependency to gitlab-shell@v1.9.8. This version of
gitlab-shell in turn depends on gitaly@v1.68.0 which depends on
cloudflare/tableflip@v0.0.0-20190329062924-8392f1641731. This commit
adds the missing checksum for that specific revision of tableflip.
|
|
[ci skip]
|
|
Refactor Praefect test server helpers
See merge request gitlab-org/gitaly!2135
|
|
|
|
Add gitlab api access
See merge request gitlab-org/gitaly!2156
|
|
Allow port reuse in tableflip
Closes #2734
See merge request gitlab-org/gitaly!2175
|
|
|
|
Write out enable-writes response correctly
See merge request gitlab-org/gitaly!2172
|
|
Currently EnableWrites RPC is writing out a nil response, which in
turn returns an error to the client. This fixes the problem by
writing out an empty answer when there is no error.
|
|
Metrics and improved logging for the transaction manager
See merge request gitlab-org/gitaly!2165
|
|
Now that the transaction manager has a set of metrics, we need to wire
them up in Praefect. Let's do so.
|
|
We currently have no metrics in the transaction manager, making it hard
to estimate how many transaction succeed, how many fail and how long a
node has to wait until a quorum was reached and the transaction is
committed. This commit thus adds two metrics, one to count the number of
registered, started, invalid and committed transactions and one to
measure the delay between start and finish of the voting process.
|
|
We currently lock the complete transaction manager during the whole
lifetime of `mgr.StartTransaction()`, even though we only need the
lock in a single place where we access the map of transactions. Let's
thus reduce the scope of the lock to only guard the map access to
improve efficiency.
|
|
The log message in case a transaction is committed is missing the "node"
field right now. In case we ever decide we want to enable transactions
in case a quorum was reached, then a subset of nodes will commit the
transaction while another subset will abort it. Thus, having the "node"
field logged in such a case may help quite a lot to see what's going on.
|
|
While we already log a transaction that is being started as well as a
successful transaction, we do not yet any error cases. Refactor the code
to put verification of transactions into its own function to unify all
current error cases into a single path and log any returned errors.
|
|
Support bindir in Makefile
Closes gitlab#213467
See merge request gitlab-org/gitaly!2167
|
|
Configure logging before running sub-commands
Closes #2500
See merge request gitlab-org/gitaly!2169
|
|
|
|
|
|
Praefect: primary election must support multi virtual storages
See merge request gitlab-org/gitaly!2170
|
|
Election of the new primary node must take into account
relation of the node to the proper virtual storage.
To speed up SQL operations index were introduced based on
'virtual_storage' and 'target_node_storage' values.
As an addition down migration was fixed to remove new type.
|
|
Remove unnecessary return clause causing lint failures
See merge request gitlab-org/gitaly!2161
|
