| Age | Commit message (Collapse) | Author |
|---|
|
|
|
[ci skip]
|
|
[ci skip]
|
|
Add HTTP Host to all requests that use URLs
See merge request gitlab-org/security/gitaly!47
|
|
|
|
|
|
[ci skip]
|
|
[ci skip]
|
|
git: Disallow use of replace refs
See merge request gitlab-org/security/gitaly!42
|
|
|
|
Optimize link repository ID migration (14.5)
See merge request gitlab-org/gitaly!4237
|
|
gitaly-lfs-smudge: Fix missing close for HTTP body (14.5)
See merge request gitlab-org/gitaly!4240
|
|
The gitaly-lfs-smudge command is a smudge filter for Git which will
replace contents of LFS pointers with the actual LFS object's contents.
To do so, we need to request the object's contents from Rails via an
HTTP request. The tests exercising this code all of a sudden started
failing due to leaking Goroutines, where the leak happens in the HTTP
handling code. And sure enough: we never close the `http.Response` body,
which may likely be the root cause here.
Fix this by always closing the body. While I have no idea why leaks
started to happen just now, chances are high that this fixes the new
flake.
|
|
The previous migration was slow at times because the update would cause
PostgreSQL to do a merge join and then filter out rows matching
`repository_id IS NULL`. As more rows migrated gained a `repository_id`,
this would increase the query time significantly for each batch.
The batching was added to deal with limiting the payload size of a
trigger update.
We can make this migration go faster by disabling the triggers in the
transactions, rollback to 2bbec66c, and re-enable the trigger.
Relates to https://gitlab.com/gitlab-org/gitaly/-/issues/3973
Changelog: fixed
|
|
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
catfile: Ensure structs are properly aligned in memory for 32-bit CPUs (14.5)
See merge request gitlab-org/gitaly!4142
|
|
In GitLab 14.5, the use of `atomic.LoadInt64` caused a `panic: unaligned
64-bit atomic operation` on a Raspberry Pi2 platform. While we don't
officially support 32-bit processors, we have been supporting armfh
builds for years now.
From https://pkg.go.dev/sync/atomic#pkg-note-BUG: On ARM, 386, and
32-bit MIPS, it is the caller's responsibility to arrange for 64-bit
alignment of 64-bit words accessed atomically. The first word in a
variable or in an allocated struct, array, or slice can be relied upon
to be 64-bit aligned.
We can fix this problem by listing the int64 and int32 atomic fields in
order to ensure they are properly aligned. This does not change the
overall size of the structure.
For example, with Object:
Before: https://go.dev/play/p/MCk9TnxhkCY (80 bytes)
After: https://go.dev/play/p/83jdBOP-wpA (80 bytes)
Relates to https://gitlab.com/gitlab-org/gitaly/-/issues/3938
Changelog: fixed
|
|
'14-5-stable'
praefect: Backport separate endpoint for datastore collector (v14.5)
See merge request gitlab-org/gitaly!4107
|
|
Our current code path will trigger the RepositoryStoreCollector to query
the database on startup, even if the prometheus listener is not
listening. This is because we call DescribeByCollect in the Describe
method. The Prometheus client will call Describe on Register, which ends
up triggering the Collect method and hence runs the queries. Instead, we
can just provide the decriptions separately from the Collect method.
Changelog: fixed
(cherry picked from commit 90cb7fb7b9f8703547fa62719650394478653c62)
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
Add /db_metrics metrics endpoint for database metrics
Closes #3286
See merge request gitlab-org/gitaly!4085
|
|
Add a test to ensure that the
PrometheusExcludeDatabaseFromDefaultMetrics controls whether or not it
gets registered in the main metrics registry.
|
|
gitaly/server: Fix race checking for expected log entries
Closes #3916
See merge request gitlab-org/gitaly!4086
|
|
demo: Activate PostgreSQL slow logs
See merge request gitlab-org/gitaly!4082
|
|
By default, when metrics are enabled, then each Praefect will expose
information about how many read-only repositories there are, which
requires Praefect to query the database. First, this will result in the
same metrics being exposed by every Praefect given that the database is
shared between all of them. And second, this will cause one query per
Praefect per scraping run. This cost does add up and generate quite some
load on the database, especially so if there is a lot of repositories in
that database, up to a point where it may overload the database
completely.
Fix this issue by splitting metrics which hit the database into a
separate endpoint "/db_metrics". This allows admins to set up a separate
scraper with a different scraping interval for this metric, and
furthermore it gives the ability to only scrape this metric for one of
the Praefect instances so the work isn't unnecessarily duplicated.
Given that this is a breaking change which will get backported, we must
make this behaviour opt-in for now. We thus include a new configuration
key "prometheus_use_database_endpoint" which enables the new behaviour
such that existing installations' metrics won't break on a simple point
release. The intent is to eventually remove this configuration though
and enable it for all setups on a major release.
Changelog: added
|
|
ruby: Clean up dead code and use temporary directory
See merge request gitlab-org/gitaly!4079
|
|
log: Flakiness in TestPayloadBytes
Closes #3918
See merge request gitlab-org/gitaly!4083
|
|
Sometimes it happens that the log contains not
enough entries in the output. It may happen because
method handling happens faster than flush of the
logger. The checks now start after we ensure all
log entries are in place by shutting down the server.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3918
|
|
The Gitaly server factory tests verify that various parts of the
wiring for a newly created server work as expected. One part we try to
verify is that the logger we pass is correctly being used. This is done
by doing a health check RPC call and then checking that we got the
expected log entry.
This is prone to a race though: the log entry we expect to exist is
created by our middleware and may thus be created after the client has
already observed the result of the RPC call. As a result, it may happen
that we do not yet have the log entry at the time of checking.
Fix this race by gracefully stopping the server such that any ongoing
requests were drained and thus fully processed.
|
|
Bump required golang to 1.16.10
See merge request gitlab-org/gitaly!4077
|
|
This commit will cause PostgreSQL to log any database statements taking
longer than 500 ms. This is useful to flag any slow queries running in
Praefect.
|
|
Remove CloneFromPool and CloneFromPoolInternal RPCs
Closes #3899
See merge request gitlab-org/gitaly!4064
|
|
Perform health check updates in an ordered manner
Closes #3907
See merge request gitlab-org/gitaly!4073
|
|
objectpool: Fix data race with logger
Closes #3914
See merge request gitlab-org/gitaly!4080
|
|
In our test to verify that we correctly collect objectpool statistics
via the logger, we create our own logger with a backing buffer. This
buffer is being accessed concurrently though, once by the gRPC
middleware which tries to write into it and once from the test itself
after the RPC call has finished to verify it contains what we expect it
to contain. Given that we don't use locking for this the result is a
data race.
Fix the data race by instead using a NullLogger and extracting the log
entries via a hook. This also allows us to tighten our checks to assert
we see all expected stats.
|
|
Track payload bytes for RPC
Closes #3867
See merge request gitlab-org/gitaly!4030
|
|
commit: Refactor recursive `GetTreeEntries()` for efficiency
Closes #3888
See merge request gitlab-org/gitaly!4052
|
|
Due to our piecemeal conversions from Ruby to Go, it's hard at times to
find Ruby parts which are not used anymore. As a result, we have
accumulated quite a lot of baggage which was missed to be removed. While
this is not much of a problem by itself, it does create tension with the
upcoming change to support deployments with bundled Git versions, where
we don't have a complete Git installation anymore but only a small set
of binaries.
Remove unused code to make the transition easier.
|
|
Test data for our rspecs is currently created inside of the worktree in
"ruby/tmp". This is bad practice, and we have long since converted our
Go tests to use temporary directories instead.
Convert rspecs to do the same and remove the corresponding ignore rule.
|
|
Gitaly contains CloneFromPool and CloneFromPoolInternal RPCs that were
added over two years ago in 495a384d41e8b4ed5a6e3e45375eda6aecd4f4fc.
They don't appear to be used anywhere in Gitaly or Rails. This commit
removes the unused RPCs.
Changelog: removed
|
|
Revert "repository: Remove deprecated and unused fields from GetRawChanges"
See merge request gitlab-org/gitaly!4078
|
|
This reverts commit f80312d8c (repository: Remove deprecated and unused
fields from GetRawChanges, 2021-11-11), which has removed `OldPath` and
`NewPath` from the RawChanges RPC's response. As it turns out, these
deprecated fields are still used by the Elasticsearch indexer. The
indexer will be converted to use new paths in v14.6 such that we can
reintroduce the removal in v14.7 in Gitaly.
|
|
sql-migrate: Update storage_repositories table
Closes #3806
See merge request gitlab-org/gitaly!4047
|
|
git: Ignore fsck errors for zero-padded filemodes
Closes #2889
See merge request gitlab-org/gitaly!4051
|
