diff options
Diffstat (limited to 'doc/administration/operations/sidekiq_memory_killer.md')
| -rw-r--r-- | doc/administration/operations/sidekiq_memory_killer.md | 58 |
1 files changed, 13 insertions, 45 deletions
diff --git a/doc/administration/operations/sidekiq_memory_killer.md b/doc/administration/operations/sidekiq_memory_killer.md index 509bc77a8c3..8eac42f2fe2 100644 --- a/doc/administration/operations/sidekiq_memory_killer.md +++ b/doc/administration/operations/sidekiq_memory_killer.md @@ -26,50 +26,18 @@ run as a process group leader (e.g., using `chpst -P`). If using Omnibus or the The MemoryKiller is controlled using environment variables. -- `SIDEKIQ_DAEMON_MEMORY_KILLER`: defaults to 0. When set to 1, the MemoryKiller - works in _daemon_ mode. Otherwise, the MemoryKiller works in _legacy_ mode. - - In _legacy_ mode, the MemoryKiller checks the Sidekiq process RSS after each job. - - In _daemon_ mode, the MemoryKiller checks the Sidekiq process RSS every 3 seconds - (defined by `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL`). - -- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is greater - than 0, the MemoryKiller is enabled. Otherwise the MemoryKiller is disabled. - - `SIDEKIQ_MEMORY_KILLER_MAX_RSS` defines the Sidekiq process allowed RSS. - - In _legacy_ mode, if the Sidekiq process exceeds the allowed RSS then an irreversible - delayed graceful restart will be triggered. The restart of Sidekiq will happen - after `SIDEKIQ_MEMORY_KILLER_GRACE_TIME` seconds. - - In _daemon_ mode, if the Sidekiq process exceeds the allowed RSS for longer than - `SIDEKIQ_MEMORY_KILLER_GRACE_TIME` the graceful restart will be triggered. If the - Sidekiq process go below the allowed RSS within `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`, - the restart will be aborted. - - The default value for Omnibus packages is set +- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is + greater than 0, then after each Sidekiq job, the MemoryKiller will check the + RSS of the Sidekiq process that executed the job. If the RSS of the Sidekiq + process (expressed in kilobytes) exceeds SIDEKIQ_MEMORY_KILLER_MAX_RSS, a + delayed shutdown is triggered. The default value for Omnibus packages is set [in the omnibus-gitlab repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). - -- `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS`: is used by _daemon_ mode. If the Sidekiq - process RSS (expressed in kilobytes) exceeds `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS`, - an immediate graceful restart of Sidekiq is triggered. - -- `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL`: used in _daemon_ mode to define how - often to check process RSS, default to 3 seconds. - -- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults to 900 seconds (15 minutes). - The usage of this variable is described as part of `SIDEKIQ_MEMORY_KILLER_MAX_RSS`. - -- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. This defines the - maximum time allowed for all Sidekiq jobs to finish. No new jobs will be accepted - during that time, and the process will exit as soon as all jobs finish. - - If jobs do not finish during that time, the MemoryKiller will interrupt all currently - running jobs by sending `SIGTERM` to the Sidekiq process. - - If the process hard shutdown/restart is not performed by Sidekiq, - the Sidekiq process will be forcefully terminated after - `Sidekiq.options[:timeout] * 2` seconds. An external supervision mechanism - (e.g. Runit) must restart Sidekiq afterwards. +- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults to 900 seconds (15 minutes). When + a shutdown is triggered, the Sidekiq process will keep working normally for + another 15 minutes. +- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace + time has expired, the MemoryKiller tells Sidekiq to stop accepting new jobs. + Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells + Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must + restart Sidekiq. |