diff options
Diffstat (limited to 'doc')
-rw-r--r-- | doc/api/epics.md | 1 | ||||
-rw-r--r-- | doc/api/features.md | 42 | ||||
-rw-r--r-- | doc/development/auto_devops.md | 6 | ||||
-rw-r--r-- | doc/development/documentation/index.md | 2 | ||||
-rw-r--r-- | doc/development/pipelines.md | 17 | ||||
-rw-r--r-- | doc/user/application_security/index.md | 16 | ||||
-rw-r--r-- | doc/user/application_security/sast/index.md | 82 | ||||
-rw-r--r-- | doc/user/project/settings/project_access_tokens.md | 2 |
8 files changed, 92 insertions, 76 deletions
diff --git a/doc/api/epics.md b/doc/api/epics.md index 0cffc1bee85..6ca6f04b741 100644 --- a/doc/api/epics.md +++ b/doc/api/epics.md @@ -67,6 +67,7 @@ GET /groups/:id/epics?state=opened | `updated_before` | datetime | no | Return epics updated on or before the given time | | `include_ancestor_groups` | boolean | no | Include epics from the requested group's ancestors. Default is `false` | | `include_descendant_groups` | boolean | no | Include epics from the requested group's descendants. Default is `true` | +| `my_reaction_emoji` | string | no | Return epics reacted by the authenticated user by the given emoji. `None` returns epics not given a reaction. `Any` returns epics given at least one reaction. Introduced in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31479)| ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics diff --git a/doc/api/features.md b/doc/api/features.md index a43f2daa93f..78457076064 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -32,6 +32,16 @@ Example response: ] }, { + "name": "my_user_feature", + "state": "on", + "gates": [ + { + "key": "percentage_of_actors", + "value": 34 + } + ] + }, + { "name": "new_library", "state": "on", "gates": [ @@ -58,6 +68,7 @@ POST /features/:name | --------- | ---- | -------- | ----------- | | `name` | string | yes | Name of the feature to create or update | | `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time | +| `key` | string | no | `percentage_of_actors` or `percentage_of_time` (default) | | `feature_group` | string | no | A Feature group name | | `user` | string | no | A GitLab username | | `group` | string | no | A GitLab group's path, for example `gitlab-org` | @@ -89,6 +100,37 @@ Example response: } ``` +### Set percentage of actors rollout + +Rollout to percentage of users. + +```plaintext +POST https://gitlab.example.com/api/v4/features/my_user_feature?private_token=<your_access_token> +Content-Type: application/x-www-form-urlencoded +value=42&key=percentage_of_actors& +``` + +Example response: + +```json +{ + "name": "my_user_feature", + "state": "conditional", + "gates": [ + { + "key": "boolean", + "value": false + }, + { + "key": "percentage_of_time", + "value": 42 + } + ] +} +``` + +Rolls out the `my_user_feature` to `42%` of users. + ## Delete a feature Removes a feature gate. Response is equal when the gate exists, or doesn't. diff --git a/doc/development/auto_devops.md b/doc/development/auto_devops.md index dba68974dd5..7d0c020ef96 100644 --- a/doc/development/auto_devops.md +++ b/doc/development/auto_devops.md @@ -1,7 +1,11 @@ # Auto DevOps development guide This document provides a development guide for contributors to -[Auto DevOps](../topics/autodevops/index.md) +[Auto DevOps](../topics/autodevops/index.md). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +An [Auto DevOps technical walk-through](https://youtu.be/G7RTLeToz9E) +is also available on YouTube. ## Development diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index ac96cfd77d8..9c42328810d 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -530,7 +530,7 @@ on or off when markdownlint was enabled on the docs. #### Vale -[Vale](https://errata-ai.github.io/vale/) is a grammar, style, and word usage linter +[Vale](https://errata-ai.gitbook.io/vale/) is a grammar, style, and word usage linter for the English language. Vale's configuration is stored in the [`.vale.ini`](https://gitlab.com/gitlab-org/gitlab/blob/master/.vale.ini) file located in the root directory of the [GitLab repository](https://gitlab.com/gitlab-org/gitlab). diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index d15ac8d5320..6f46075c711 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -70,11 +70,7 @@ that are scoped to a single [configuration parameter](../ci/yaml/README.md#confi | `.default-retry` | Allows a job to [retry](../ci/yaml/README.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. | | `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (e.g. tests). | | `.default-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails and frontend tasks. | -| `.use-pg9` | Allows a job to use the `postgres:9.6.17` and `redis:alpine` services. | -| `.use-pg10` | Allows a job to use the `postgres:10.12` and `redis:alpine` services. | | `.use-pg11` | Allows a job to use the `postgres:11.6` and `redis:alpine` services. | -| `.use-pg9-ee` | Same as `.use-pg9` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. | -| `.use-pg10-ee` | Same as `.use-pg10` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. | | `.use-pg11-ee` | Same as `.use-pg11` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. | | `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | | `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` environment variable. | @@ -159,10 +155,9 @@ request, be sure to start the `dont-interrupt-me` job before pushing. | Where? | PG version | | ------ | ------ | -| MRs | 9.6 | -| `master` (non-scheduled pipelines) | 9.6 / 11 | -| 2-hourly scheduled pipelines | 9.6 / 11 | -| Nightly scheduled pipeline | 9.6 / 10 | +| MRs | 11 | +| `master` (non-scheduled pipelines) | 11 | +| 2-hourly scheduled pipelines | 11 | ### Long-term plan @@ -215,7 +210,7 @@ graph RL; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" - 1-6["setup-test-env pg11 (9.6 minutes)"]; + 1-6["setup-test-env (9.6 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" 1-7["review-stop-failed-deployment"]; 1-8["dependency_scanning"]; @@ -329,7 +324,7 @@ graph RL; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" - 1-6["setup-test-env pg11 (9.6 minutes)"]; + 1-6["setup-test-env (9.6 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" 1-7["review-stop-failed-deployment"]; 1-8["dependency_scanning"]; @@ -468,7 +463,7 @@ graph RL; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" - 1-6["setup-test-env pg11 (9.6 minutes)"]; + 1-6["setup-test-env (9.6 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" 1-7["review-stop-failed-deployment"]; 1-8["dependency_scanning"]; diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index ee69aa0cd39..572619f7630 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -321,7 +321,8 @@ You can do it quickly by following the hyperlink given to run a new pipeline. ### Getting error message `sast job: stage parameter should be [some stage name here]` -When including a security job template like [`SAST`](sast/index.md#configuration), +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: ```plaintext @@ -334,22 +335,23 @@ This error appears when the included job's stage (named `test`) isn't declared i To fix this issue, you can either: - Add a `test` stage in your `.gitlab-ci.yml`. -- Change the default stage of the included security jobs. For example, with `SAST`: +- Change the default stage of the included security jobs. For example, with SpotBugs (SAST): ```yaml include: template: SAST.gitlab-ci.yml - sast: + spotbugs-sast: stage: unit-tests ``` -[Learn more on overriding the SAST template](sast/index.md#overriding-the-sast-template). +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). All the security scanning tools define their stage, so this error can occur with all of them. ### Getting error message `sast job: config key may not be used with 'rules': only/except` -When including a security job template like [`SAST`](sast/index.md#overriding-the-sast-template), +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: ```plaintext @@ -358,14 +360,14 @@ Found errors in your .gitlab-ci.yml: jobs:sast config key may not be used with `rules`: only/except ``` -This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-the-sast-template) +This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#onlyexcept-basic) To fix this issue, you must either: - [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). - (Temporarily) [Pin your templates to the deprecated versions](#pin-your-templates-to-the-deprecated-versions) -[Learn more on overriding the SAST template](sast/index.md#overriding-the-sast-template). +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). #### Transitioning your `only/except` syntax to `rules` diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 698a96bf607..cd713db0797 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -48,14 +48,12 @@ A pipeline consists of multiple jobs, including SAST and DAST scanning. If any j ## Requirements -To run a SAST job, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +To run SAST jobs, by default, you need GitLab Runner with the +[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +If you're using the shared Runners on GitLab.com, this is enabled by default. -Privileged mode is not necessary if you've [disabled Docker in Docker -for SAST](#disabling-docker-in-docker-for-sast). +Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker). CAUTION: **Caution:** Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. @@ -114,15 +112,13 @@ include: - template: SAST.gitlab-ci.yml ``` -The included template will create a `sast` job in your CI/CD pipeline and scan +The included template will create SAST jobs in your CI/CD pipeline and scan your project's source code for possible vulnerabilities. The results will be saved as a [SAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssast-ultimate) that you can later download and analyze. Due to implementation limitations, we -always take the latest SAST artifact available. Behind the scenes, the -[GitLab SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) -is used to detect the languages or frameworks used, and in turn runs the matching scan tools. +always take the latest SAST artifact available. ### Customizing the SAST settings @@ -143,23 +139,24 @@ variables: Because the template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable takes precedence. -### Overriding the SAST template +### Overriding SAST jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `sast` job after the -template inclusion and specify any additional keys under it. For example: +If you want to override a job definition (for example, change properties like +`variables` or `dependencies`), you need to declare a job with the same name as the SAST job to override, after the +template inclusion and specify any additional keys under it. +For example, this enables `FAIL_NEVER` for the `spotbugs` analyzer: ```yaml include: - template: SAST.gitlab-ci.yml -sast: +spotbugs-sast: variables: - CI_DEBUG_TRACE: "true" + FAIL_NEVER: 1 ``` ### Using environment variables to pass credentials for private repositories @@ -177,44 +174,27 @@ you can use the `MAVEN_CLI_OPTS` environment variable. Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Disabling Docker in Docker for SAST +### Enabling Docker-in-Docker -You can avoid the need for Docker in Docker by running the individual analyzers. -This does not require running the executor in privileged mode. For example: +If needed, you can restore the behavior of SAST prior to %13.0 by enabling back Docker-in-Docker. +You need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode), and the variable `SAST_DISABLE_DIND` set to `false`: ```yaml include: - template: SAST.gitlab-ci.yml variables: - SAST_DISABLE_DIND: "true" + SAST_DISABLE_DIND: "false" ``` -This will create individual `<analyzer-name>-sast` jobs for each analyzer that runs in your CI/CD pipeline. - -By removing Docker-in-Docker (DIND), GitLab relies on [Linguist](https://github.com/github/linguist) -to start relevant analyzers depending on the detected repository language(s) instead of the -[orchestrator](https://gitlab.com/gitlab-org/security-products/dependency-scanning/). However, there -are some differences in the way repository languages are detected between DIND and non-DIND. You can -observe these differences by checking both Linguist and the common library. For instance, Linguist -looks for `*.java` files to spin up the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) -image, while orchestrator only looks for the existence of `pom.xml`, `build.xml`, `gradlew`, -`grailsw`, or `mvnw`. GitLab uses Linguist to detect new file types in the default branch. -When introducing files or dependencies for a new language or package manager, the -corresponding scans won't be triggered in the MR, and will only run on the default branch once the -MR is merged. This will be addressed by [#211702](https://gitlab.com/gitlab-org/gitlab/-/issues/211702). - -NOTE: **Note:** -With the current language detection logic, any new languages or frameworks introduced within the -context of a merge request don't trigger a corresponding scan. These scans only occur once the code -is committed to the default branch. +This will create a single `sast` job in your CI/CD pipeline +instead of multiple `<analyzer-name>-sast` jobs. #### Enabling Kubesec analyzer > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12752) in GitLab Ultimate 12.6. -When [Docker in Docker is disabled](#disabling-docker-in-docker-for-sast), -you will need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the +You need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the Kubesec analyzer. In `.gitlab-ci.yml`, define: ```yaml @@ -222,7 +202,6 @@ include: - template: SAST.gitlab-ci.yml variables: - SAST_DISABLE_DIND: "true" SCAN_KUBERNETES_MANIFESTS: "true" ``` @@ -248,9 +227,6 @@ stages: include: - template: SAST.gitlab-ci.yml -variables: - SAST_DISABLE_DIND: "true" - build: stage: build script: @@ -297,7 +273,7 @@ The following are Docker image-related variables. | `SAST_ANALYZER_IMAGE_PREFIX` | **DEPRECATED**: Use `SECURE_ANALYZERS_PREFIX` instead. | | `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | -| `SAST_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-sast). | +| `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | #### Vulnerability filters @@ -317,26 +293,23 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre #### Docker-in-Docker orchestrator -The following variables configure the Docker-in-Docker orchestrator. +The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). | Environment variable | Default value | Description | |------------------------------------------|---------------|-------------| -| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | -| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | +| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). | | `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | | `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | | `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`.| -NOTE: **Note:** -Timeout variables are not applicable for setups with [disabled Docker In Docker](index.md#disabling-docker-in-docker-for-sast). - #### Analyzer settings Some analyzers can be customized with environment variables. | Environment variable | Analyzer | Description | |-----------------------------|----------|-------------| -| `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests when [Docker in Docker](#disabling-docker-in-docker-for-sast) is disabled. | +| `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | | `ANT_HOME` | SpotBugs | The `ANT_HOME` environment variable. | | `ANT_PATH` | SpotBugs | Path to the `ant` executable. | | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | @@ -521,7 +494,7 @@ run successfully. For more information, see [Offline environments](../offline_de To use SAST in an offline environment, you need: -- To [disable Docker-In-Docker](#disabling-docker-in-docker-for-sast). +- To keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - Docker Container Registry with locally available copies of SAST [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. @@ -577,7 +550,6 @@ include: variables: SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" - SAST_DISABLE_DIND: "true" ``` The SAST job should now use local copies of the SAST analyzers to scan your code and generate diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 7f92d0c59fe..303a6f6d3be 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -47,7 +47,7 @@ the following table. | Scope | Description | | ------------------ | ----------- | -| `api` | Grants complete read/write access to scoped project API. | +| `api` | Grants complete read/write access to the scoped project API. | | `read_api` | Grants read access to the scoped project API. | | `read_registry` | Allows read-access (pull) to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | | `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | |