| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Detect if pipeline runs for a GitHub pull request
When using a mirror for CI/CD only we register a pull_request
webhook. When a pull_request webhook is received, if the
source branch SHA matches the actual head of the branch in the
repository we create immediately a new pipeline for the
external pull request. Otherwise we store the
pull request info for when the push webhook is received.
When using "only/except: external_pull_requests" we can detect
if the pipeline has a open pull request on GitHub and create or
not the job based on that.
|
|
|
|
Reduce dedup calls to gc only
See merge request gitlab-org/gitlab-ce!32083
|
|
|
|
This commits avoiding conflicts between ArchiveTraceWorker
and ArchiveTracesCronWorker by changing the target of the
latter worker.
|
|
Look up upstream commits once before queuing ProcessCommitWorkers
Closes #65464
See merge request gitlab-org/gitlab-ce!31871
|
|
Previously `ProjectCacheWorker` would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. `ProjectCacheWorker` would expire
three items:
1. Repository size: This only needs to be updated once per push.
2. Commit count: This only needs to be updated if the default branch
is updated.
3. Project method caches: This only needs to be updated if the default
branch changes, but only if certain files change (e.g. README,
CHANGELOG, etc.).
Because the third item requires looking at the actual changes in the
commit deltas, we schedule one `ProjectCacheWorker` to handle the first
two cases, and schedule a separate `ProjectCacheWorker` for the third
case if it is needed. As a result, this brings down the number of
`ProjectCacheWorker` jobs from N to 2.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
|
|
Instead of checking if a commit already exists in the upstream project
in its ProcessCommitWorker and bailing out if it does, we check the
existence of all commits in bulk in Git::BranchHooksService, so that we
can skip scheduling ProcessCommitWorker jobs for those commits
that already exist upstream entirely.
|
|
Source Code Usage Ping for Create SMAU
|
|
**Prevention of running 2 simultaneous updates**
Instead of using `RemoteMirror#update_status` and raise an error if
it's already running to prevent the same mirror being updated at the
same time we now use `Gitlab::ExclusiveLease` for that.
When we fail to obtain a lease in 3 tries, 30 seconds apart, we bail
and reschedule. We'll reschedule faster for the protected branches.
If the mirror already ran since it was scheduled, the job will be
skipped.
**Error handling: Remote side**
When an update fails because of a `Gitlab::Git::CommandError`, we
won't track this error in sentry, this could be on the remote side:
for example when branches have diverged.
In this case, we'll try 3 times scheduled 1 or 5 minutes apart.
In between, the mirror is marked as "to_retry", the error would be
visible to the user when they visit the settings page.
After 3 tries we'll mark the mirror as failed and notify the user.
We won't track this error in sentry, as it's not likely we can help
it.
The next event that would trigger a new refresh.
**Error handling: our side**
If an unexpected error occurs, we mark the mirror as failed, but we'd
still retry the job based on the regular sidekiq retries with
backoff. Same as we used to
The error would be reported in sentry, since its likely we need to do
something about it.
|
|
Previously each tag in a push would invoke the Gitaly `FindAllTags` RPC
since the tag cache would be invalidated with every tag.
We can eliminate those extraneous calls by expiring the tag cache once
in `PostReceive` and taking advantage of the cached tags.
Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/65795
|
|
Only expire branch cache once per push
See merge request gitlab-org/gitlab-ce!31653
|
|
We migrated all logic to `PipelineProcessWorker`
and this worker become redundant.
|
|
|
|
Whenever `PostReceive` is enqueued, `UpdateMergeRequestsWorker`
is enqueued and `MergeRequests::RefreshService` is called, it'll
check if the source branch of each MR asssociated to the push exists
or not via `MergeRequest#source_branch_exists?`. The said method will
call `Repository#branch_exists?` which is cached in `Rails.cache`.
When the cache contains outdated data and the source branch actually
exists, the `MergeRequests#RefreshService` job will close associated
MRs which is not correct.
The fix is to expire the branches cache of the project so we have
updated data during the post receive hook which will help in the
accuracy of the check if we need to close associated MRs or not.
|
|
We added retry_count increment when job is recovered
|
|
Extend PipelineProcessWorker to accept a list of builds
See merge request gitlab-org/gitlab-ce!31425
|
|
This changes used worker from `BuildProcessWorker`
to `PipelineProcessWorker` to make pipeline
processing much simpler. We process `pipeline_id`,
based on some triggers.
|
|
After measuring the impact of the namespace storage on
https://gitlab.com/gitlab-org/gitlab-ce/issues/64092. It was decided
that it's performant enough. So we can freely remove the feature flag
Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/64092
|
|
This implements the support for `needs:` keyword
as part of GitLab CI. That makes some of the jobs
to be run out of order.
|
|
We dropped MySQL support and a lot of mysql specific code has been
removed in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29608.
This comes in from the other direction and removes any `if postgresql?`
branches.
|
|
Also removes all logic from ClusterConfigureWorker
and ClusterProjectConfigureWorker, which are also
no longer used.
|
|
Often live traces are removed even though the archived trace
doesn't exist. This commit checkes the existence strictly.
|
|
Adds chaos endpoints to Sidekiq
Closes #64663
See merge request gitlab-org/gitlab-ce!30814
|
|
This allows the chaos endpoints to be invoked in Sidekiq so that this
environment can be tested for resilience.
|
|
* Splay the start by sleeping up to one minute.
* Adjust sideikiq retries to try up to approximately three hours.
* Randomize the minute fully within the hour.
* Psudo-randomize the day of the week based on the instance UUID.
|
|
* remove feature flag for admin settings
* remove feature flag for domain settings
|
|
Suggests to use a JSON structured log instead
Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/54102
|
|
|
|
This MR introduces tracking of the `rebase_jid` for merge requests. As
with `merge_ongoing?`, `rebase_in_progress?` will now return true if a
rebase is proceeding in sidekiq.
After one release, we should remove the Gitaly-based lookup of rebases.
It is much better to track this kind of thing via the database.
|
|
- Add two new ActiveRecord models:
- RootNamespaceStoragestatistics will persist root namespace statistics
- NamespaceAggregationSchedule will save information when a new update
to the namespace statistics needs to be scheduled
- Inject into UpdateProjectStatistics concern a new callback that will
call an async job to insert a new row onto NamespaceAggregationSchedule
table
- When a new row is inserted a new job is scheduled. This job will
update call an specific service to update the statistics and after that
it will delete thee aggregated scheduled row
- The RefresherServices makes heavy use of arel to build composable
queries to update Namespace::RootStorageStatistics attributes.
- Add an extra worker to traverse pending rows on
NAmespace::AggregationSchedule table and schedule a worker for each one
of this rows.
- Add an extra worker to traverse pending rows on
NAmespace::AggregationSchedule table and schedule a worker for each one
of this rows
|
|
Also add ::Gitlab::LetsEncrypt.enabled? shortcut
and simplify it a lot
|
