Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2023-05-19 06:08:19 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2023-05-19 06:08:19 +0300
commitec1f0dd7f46a6d25e4a47c8254ac67b720f81e89 (patch)
treecea26c43c6a48b785eba0e97a20592e4545028e1 /doc/development
parent1f9bddaf87f0b6c93a9617bad0ae731baf16d268 (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/development')
-rw-r--r--doc/development/ai_features.md6
-rw-r--r--doc/development/feature_flags/index.md40
-rw-r--r--doc/development/sec/analyzer_development_guide.md6
3 files changed, 49 insertions, 3 deletions
diff --git a/doc/development/ai_features.md b/doc/development/ai_features.md
index 25da99addc1..f8d19dc26bf 100644
--- a/doc/development/ai_features.md
+++ b/doc/development/ai_features.md
@@ -66,7 +66,9 @@ All AI features are experimental.
1. Enable **Experiment features**
1. Enable **Third-party AI services**
1. Enable the specific feature flag for the feature you want to test
-1. Set either the required access token `OpenAi` or `Vertex`. Ask in [`#ai_enablement_team`](https://gitlab.slack.com/archives/C051K31F30R) to receive an access token.
+1. Set the required access token. To receive an access token:
+ 1. For Vertex, follow the [instructions below](#configure-gcp-vertex-access).
+ 1. For all other providers, create an access request where `@m_gill`, `@wayne`, and `@timzallmann` are the tech stack owners.
### Set up the embedding database
@@ -102,7 +104,7 @@ tail -f log/llm.log
In order to obtain a GCP service key for local development, please follow the steps below:
-- Create a sandbox GCP environment by visiting [this page](https://about.gitlab.com/handbook/infrastructure-standards/#individual-environment) and following the instructions
+- Create a sandbox GCP environment by visiting [this page](https://about.gitlab.com/handbook/infrastructure-standards/#individual-environment) and following the instructions, or by requesting access to our existing group environment by using [this template](https://gitlab.com/gitlab-com/it/infra/issue-tracker/-/issues/new?issuable_template=gcp_group_account_iam_update_request). At this time, access to any endpoints outside of `text-bison` or `chat-bison` must be made through the group environment.
- In the GCP console, go to `IAM & Admin` > `Service Accounts` and click on the "Create new service account" button
- Name the service account something specific to what you're using it for. Select Create and Continue. Under `Grant this service account access to project`, select the role `Vertex AI User`. Select `Continue` then `Done`
- Select your new service account and `Manage keys` > `Add Key` > `Create new key`. This will download the **private** JSON credentials for your service account. Your full settings should then be:
diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md
index c2026ab8966..4b0ad30e3ad 100644
--- a/doc/development/feature_flags/index.md
+++ b/doc/development/feature_flags/index.md
@@ -149,7 +149,8 @@ created using the [Experiment Tracking template](https://gitlab.com/gitlab-org/g
`worker` feature flags are used for controlling Sidekiq workers behavior, such as deferring Sidekiq jobs.
`worker` feature flags likely do not have any YAML definition as the name could be dynamically generated using
-the worker name itself, e.g. `defer_sidekiq_jobs:AuthorizedProjectsWorker`.
+the worker name itself, e.g. `defer_sidekiq_jobs:AuthorizedProjectsWorker`. Some examples for using `worker` type feature
+flags can be found in [deferring Sidekiq jobs](#deferring-sidekiq-jobs).
## Feature flag definition and validation
@@ -705,3 +706,40 @@ test is written to enable/disable a feature flag explicitly.
When a feature flag is changed on Staging or on GitLab.com, a Slack message will be posted to the `#qa-staging` or `#qa-production` channels to inform
the pipeline triage DRI so that they can more easily determine if any failures are related to a feature flag change. However, if you are working on a change you can
help to avoid unexpected failures by [confirming that the end-to-end tests pass with a feature flag enabled.](../testing_guide/end_to_end/feature_flags.md#confirming-that-end-to-end-tests-pass-with-a-feature-flag-enabled)
+
+## Controlling Sidekiq worker behavior with feature flags
+
+Feature flags with [`worker` type](#worker-type) can be used to control the behavior of a Sidekiq worker.
+
+### Deferring Sidekiq jobs
+
+Feature flags with the format of `defer_sidekiq_jobs:{WorkerName}` delay the execution of the worker
+by scheduling the job at a later time.
+Deferring jobs can be useful during an incident where contentious behavior from
+worker instances are saturating infrastructure resources (such as database and database connection pool).
+The implementation can be found at [DeferJobs Sidekiq server middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/sidekiq_middleware/defer_jobs.rb).
+
+NOTE:
+Jobs are deferred indefinitely as long as the feature flag is enabled. It is important to disable the
+feature flag after the worker is deemed safe to continue processing.
+
+When set to true, 100% of the jobs are deferred. When you want processing to resume, you can
+use a **percentage of time** rollout. For example:
+
+```shell
+# defer 100% of the jobs
+/chatops run feature set defer_sidekiq_jobs:SlowRunningWorker true
+
+# defer 99% of the jobs, only letting 1% processed
+/chatops run feature set defer_sidekiq_jobs:SlowRunningWorker 99
+
+# defer 50% of the jobs
+/chatops run feature set defer_sidekiq_jobs:SlowRunningWorker 50
+
+# stop deferring the jobs, jobs are being processed normally
+/chatops run feature set defer_sidekiq_jobs:SlowRunningWorker false
+```
+
+NOTE:
+The percentage of time value denotes the percentage of time the jobs are being deferred (instead of being processed).
+For example, setting to `99` means only 1% of the jobs are being processed at random.
diff --git a/doc/development/sec/analyzer_development_guide.md b/doc/development/sec/analyzer_development_guide.md
index 31058427b55..3f1420b8d3d 100644
--- a/doc/development/sec/analyzer_development_guide.md
+++ b/doc/development/sec/analyzer_development_guide.md
@@ -361,3 +361,9 @@ This issue will guide you through the whole release process. In general, you hav
- Max role: `Developer`
- Scope of the associated `GITLAB_TOKEN`:
- Expiry Date of the associated `GITLAB_TOKEN`:
+
+#### Dependency updates
+
+All dependencies and upstream scanners (if any) used in the analyzer source are updated on a monthly cadence which primarily includes security fixes and non-breaking changes.
+
+- Static Analysis team uses a custom internal tool ([SastBot](https://gitlab.com/gitlab-org/security-products/analyzers/sast-analyzer-deps-bot#dependency-update-automation)) to automate dependency management of all the SAST analyzers. SastBot generates MRs on the **8th of each month** and distributes their assignment among Static Analysis team members to take them forward for review. For details on the process, see [Dependency Update Automation](https://gitlab.com/gitlab-org/security-products/analyzers/sast-analyzer-deps-bot#dependency-update-automation).