Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2020-10-26 21:08:27 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2020-10-26 21:08:27 +0300
commita683d38a36aadac5c244299d37325424d85fa9e5 (patch)
tree781918595397211ce4e6adb4757f6c54bcd902c6 /doc
parenta1908447b77a6d005a77f1be2f717cd35274c82d (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r--doc/development/sidekiq_style_guide.md63
1 files changed, 56 insertions, 7 deletions
diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md
index 24570cfc07b..b341c31e21d 100644
--- a/doc/development/sidekiq_style_guide.md
+++ b/doc/development/sidekiq_style_guide.md
@@ -165,6 +165,22 @@ job. The work is skipped because the same work would be
done by the job that was scheduled first; by the time the second
job executed, the first job would do nothing.
+#### Strategies
+
+GitLab supports two deduplication strategies:
+
+- `until_executing`
+- `until_executed`
+
+More [deduplication strategies have been
+suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If
+you are implementing a worker that could benefit from a different
+strategy, please comment in the issue.
+
+##### Until Executing
+
+This strategy takes a lock when a job is added to the queue, and removes that lock before the job starts.
+
For example, `AuthorizedProjectsWorker` takes a user ID. When the
worker runs, it recalculates a user's authorizations. GitLab schedules
this job each time an action potentially changes a user's
@@ -173,10 +189,47 @@ same time, the second job can be skipped if the first job hasn't
begun, because when the first job runs, it creates the
authorizations for both projects.
+```ruby
+module AuthorizedProjectUpdate
+ class UserRefreshOverUserRangeWorker
+ include ApplicationWorker
+
+ deduplicate :until_executing
+ idempotent!
+
+ # ...
+ end
+end
+```
+
+##### Until Executed
+
+This strategy takes a lock when a job is added to the queue, and removes that lock after the job finishes.
+It can be used to prevent jobs from running simultaneously multiple times.
+
+```ruby
+module Ci
+ class BuildTraceChunkFlushWorker
+ include ApplicationWorker
+
+ deduplicate :until_executed
+ idempotent!
+
+ # ...
+ end
+end
+```
+
+#### Scheduling jobs in the future
+
GitLab doesn't skip jobs scheduled in the future, as we assume that
the state will have changed by the time the job is scheduled to
-execute. If you do want to deduplicate jobs scheduled in the future
-this can be specified on the worker as follows:
+execute. Deduplication of jobs scheduled in the feature is possible
+for both `until_executed` and `until_executing` strategies.
+
+If you do want to deduplicate jobs scheduled in the future,
+this can be specified on the worker by passing `including_scheduled: true` argument
+when defining deduplication strategy:
```ruby
module AuthorizedProjectUpdate
@@ -191,11 +244,7 @@ module AuthorizedProjectUpdate
end
```
-This strategy is called `until_executing`. More [deduplication
-strategies have been
-suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If
-you are implementing a worker that could benefit from a different
-strategy, please comment in the issue.
+#### Troubleshooting
If the automatic deduplication were to cause issues in certain
queues. This can be temporarily disabled by enabling a feature flag