| Age | Commit message (Collapse) | Author |
|---|
|
go: Update module go.uber.org/goleak to v1.2.1
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5409
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
go: Update module github.com/golang-jwt/jwt/v4 to v4.5.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5405
Merged-by: Will Chandler <wchandler@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
|
|
|
|
go: Update github.com/ProtonMail/go-crypto digest to 81033d7
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5394
Merged-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
go: Update module github.com/jackc/pgx/v4 to v4.18.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5390
Merged-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Sami Hiltunen <shiltunen@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
go: Update golang.org/x/exp digest to 5e25df0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5389
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
go: Update module github.com/jackc/pgtype to v1.14.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5381
Merged-by: karthik nayak <knayak@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
go: Update golang.org/x/exp digest to 46f607a
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5374
Merged-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
go: Update github.com/ProtonMail/go-crypto digest to d1d05f4
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5369
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
go: Update grpc to v1.53.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5372
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
|
|
go: Update module github.com/getsentry/sentry-go to v0.18.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5370
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
Update grpc-go to v1.53.0. This requires a small adjustment to
`TestDialNodes` as the error message format for failed connections has
changed slightly.
|
|
|
|
testhelper: Skip quarantined tests
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5362
Merged-by: John Cai <jcai@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Reviewed-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Reviewed-by: John Cai <jcai@gitlab.com>
Co-authored-by: Justin Tobler <jtobler@gitlab.com>
|
|
go: Update module github.com/rubenv/sql-migrate to v1.3.1
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5355
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
Currently there is not a standardized approach to quarantining flaky
tests. This change implements `testhelper.SkipQuarantinedTest` which can
be invoked to skip specified tests during execution.
|
|
|
|
|
|
|
|
go: Fix dependencies which have been inadvertently downgraded
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5354
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
|
|
go: Update github.com/ProtonMail/go-crypto digest to a47887b
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5273
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
With 8a29ef511 (Implement basic transaction processing with write-ahead
logging, 2022-10-14) we have implemented the basic logic for write-ahead
logging of references. As this MR introduced a bunch of logic it has
been taking a comparatively long time to get it reviewed and landed. As
part of that there had seemingly be a merge conflict in the `go.mod`
file that was mis-merged and that caused us to downgrade the following
dependencies:
- github.com/getsentry/sentry-go 0.17.0 => 0.16.0
- github.com/rubenv/sql-migrate 1.3.0 => 1.2.0
- golang.org/x/sys 0.4.0 => 0.3.0
- golang.org/x/time 0.3.0 => 0.2.0
Revert these downgrades so that we're using the intended dependency
versions again.
Changelog: fixed
|
|
gRPC supports a built-in DNS resolver. This resolver works quite well
in most scenarios. It has some drawbacks:
- After the DNS is resolved for the first time, the resolver does not
refresh the list of addresses until the client connection triggers the
resolver actively. Client connection does so when it detects some of its
subchannels are unavailable permanently. It means as soon as the client
connection is stable, the client is not aware of new hosts added to
the cluster via DNS service discovery. This behavior leads to unexpected
stickiness and workload skew, especially after a failover.
- The support for SRV record is in a weird state. This type of record
is only supported when grpclb load balancing strategy is enabled. This
strategy is deprecated, unfortunately. Its behavior is also not as we
expected. In short-term, we would like to use round-robin strategy. In
longer term, we may have a custom strategy for Raft-based cluster. Thus,
SRV service discovery is crucial in the future.
- The resolver detects service config via TXT record if any. While this
option is convenient for a generic grpc setting, it does not make sense
for Gitaly. So, we should get rid of it.
This commit implements a custom DNS resolver. This resolver has some
major features:
- Resolve DNS service discovery via A records
- Periodically refresh the DNS (5 minutes by default)
- Update DNS state only if it detects real changes
- Support logging
Service discovery via SRV records is not supported in this version to
keep the backward compatibility with Ruby clients.
|
|
go: Update module google.golang.org/grpc to v1.52.3
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5297
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
|
|
|
|
go: Update module google.golang.org/grpc to v1.52.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5256
Merged-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
go: Update github.com/ProtonMail/go-crypto digest to 068501e
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5253
Merged-by: Will Chandler <wchandler@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
Gitaly is currently lacking in transaction control. Each write coming
in launches their own Git commands which operate on the repository
concurrently. This makes transaction management difficult. It's
difficult to optimize the writes as they are being done from multiple
locations without synchronization. The concurrent writers may step on
each others toes and surface lock conflicts to the users. Recovering from
crashes is also difficult as Gitaly is not logging the modifications it is
about to perform and thus loses the transaction state on crashes. There's
also no clear notion of ordering which further complicates replication
related matters. It's not easy to say which writes a repository is missing
and which not.
We've recently designed a new replication architecture for Gitaly. The
new architecture relies on a replicated write-ahead log. The write-ahead
log defines a clear order of writes and aids in crash recovery. A single
writer will be operating on a repository which makes further optimizations
such as write batching easier.
This commit implements the first steps towards the new architecture by
implementing the TransactionManager. The TransactionManager will be
responsible for transaction management of a single repository. It will
be the single goroutine that writes into a repository and is invoked by
all other locations in the code that wish to write. It will also be
responsible for synchronizing reads by ensuring they see the changes they
are supposed to see.
TransactionManager implementation introduced here does not contain the
full implementation but aims to provide a basis for future iteration. For
now, it implements basic write processing with a write-ahead log. It
processes writes one-by-one by verifying references, logging the changes
and finally applying the changes to the repository. It also supports
recovering from the write-ahead log should the log processing be interrupted.
The reference verification behavior can be tuned on a per transaction level
to match behavior Git's `--atomic` or `--force` push flags.
The TransactionManager stores the state related to the write-ahead log in
BadgerDB, which is a key-value store that will be local to each Gitaly
storage. The values are marshaled protocol buffer messages.
This iteration is mostly concerned with the reference updating logic.
Pack files are not handled yet as aren't the internal references they need.
Symbolic references, namely for updating the default branch, are not handled
yet either. The writes are processed one by one and are acknowledged after
applying them to the repository. Given that, there's not separate logic needed
for read synchronization yet either.
The goal here is to set the initial interface and log processing, and to lock
down the reference updating logic with tests so we can later on safely start
iterating on the internals of the TransactionManager and start adding support
for the missing functionality.
|
|
|
|
|
|
|
|
|
|
go: Update module golang.org/x/time to v0.3.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5214
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
version: Make builds deterministic by dropping build time
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5201
Merged-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
|
|
Bump the LabKit dependency to v1.17. It only includes a single change,
which is a new option for the monitoring infrastructure to obtain build
information from the new `debug.BuildInfo` structure introduced with Go
1.18.
|
|
|
|
|
|
go: Update module golang.org/x/sys to v0.2.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5023
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
