diff options
Diffstat (limited to 'doc/user/application_security/secret_detection/index.md')
| -rw-r--r-- | doc/user/application_security/secret_detection/index.md | 71 |
1 files changed, 43 insertions, 28 deletions
diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 0ae038924ec..98177e804f3 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -86,14 +86,14 @@ However not all features are available on every tier. See the breakdown below fo Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), as shown in the following table: -| Capability | In Core | In Ultimate | -|:--------------------------------------------------------------------------|:--------------------|:-------------------| -| [Configure Secret Detection Scanners](#configuration) | **{check-circle}** | **{check-circle}** | -| [Customize Secret Detection Settings](#customizing-settings) | **{check-circle}** | **{check-circle}** | -| View [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** | **{check-circle}** | -| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | +| Capability | In Free | In Ultimate | +|:----------------------------------------------------------------|:--------------------|:-------------------| +| [Configure Secret Detection Scanners](#configuration) | **{check-circle}** | **{check-circle}** | +| [Customize Secret Detection Settings](#customizing-settings) | **{check-circle}** | **{check-circle}** | +| View [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** | **{check-circle}** | +| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | | [Interaction with Vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Access to Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | +| [Access to Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | ## Configuration @@ -148,7 +148,7 @@ Third party cloud and SaaS providers can [express integration interest by fillin ### Customizing settings -The Secret Detection scan settings can be changed through [environment variables](#available-variables) +The Secret Detection scan settings can be changed through [CI/CD variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. @@ -156,8 +156,21 @@ To override a job definition, (for example, change properties like `variables` o declare a job with the same name as the SAST job to override. Place this new job after the template inclusion and specify any additional keys under it. +WARNING: +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + +#### GIT_DEPTH + +The [`GIT_DEPTH` CI/CD variable](../../../ci/runners/README.md#shallow-cloning) affects Secret Detection. +The Secret Detection analyzer relies on generating patches between commits to scan content for +secrets. If you override the default, ensure the value is greater than 1. If the number of commits +in an MR is greater than the GIT_DEPTH value, Secret Detection will [fail to detect secrets](#error-couldnt-run-the-gitleaks-command-exit-status-2). + +#### Custom settings example + In the following example, we include the Secret Detection template and at the same time we -override the `secret_detection` job with the `SECRET_DETECTION_HISTORIC_SCAN` variable to `true`: +override the `secret_detection` job with the `SECRET_DETECTION_HISTORIC_SCAN` CI/CD variable to `true`: ```yaml include: @@ -171,20 +184,16 @@ secret_detection: Because the template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable takes precedence. -WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. - #### Available variables -Secret Detection can be customized by defining available variables: +Secret Detection can be customized by defining available CI/CD variables: -| Environment variable | Default value | Description | -|-------------------------|---------------|-------------| -| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | -| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | -| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | -| `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +| CI/CD variable | Default value | Description | +|-----------------------------------|---------------|-------------| +| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | +| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | +| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | +| `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | ### Custom rulesets **(ULTIMATE)** @@ -231,7 +240,7 @@ To create a custom ruleset: ### Logging level -To control the verbosity of logs set the `SECURE_LOG_LEVEL` environment variable. Messages of this logging level or higher are output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. +To control the verbosity of logs set the `SECURE_LOG_LEVEL` CI/CD variable. Messages of this logging level or higher are output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. From highest to lowest severity, the logging levels are: @@ -246,7 +255,7 @@ From highest to lowest severity, the logging levels are: GitLab 12.11 introduced support for scanning the full history of a repository. This new functionality is particularly useful when you are enabling Secret Detection in a repository for the first time and you want to perform a full secret scan. Running a secret scan on the full history can take a long time, -especially for larger repositories with lengthy Git histories. We recommend not setting this variable +especially for larger repositories with lengthy Git histories. We recommend not setting this CI/CD variable as part of your normal job definition. A new configuration variable ([`SECRET_DETECTION_HISTORIC_SCAN`](../sast/#vulnerability-filters)) @@ -307,7 +316,7 @@ Support for custom certificate authorities was introduced in the following versi | -------- | ------- | | secrets | [v3.0.0](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/releases/v3.0.0) | -### Set Secret Detection CI job variables to use local Secret Detection analyzer +### Set Secret Detection CI/CD variables to use local Secret Detection analyzer Add the following configuration to your `.gitlab-ci.yml` file. You must replace `SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: @@ -331,11 +340,15 @@ For information on this, see the [general Application Security troubleshooting s ### Error: `Couldn't run the gitleaks command: exit status 2` -This error is usually caused by the `GIT_DEPTH` value of 50 that is set for all [projects by default](../../../ci/pipelines/settings.md#git-shallow-clone). - -For example, if a pipeline is triggered from a Merge Request containing 60 commits while the `GIT_DEPTH` is set to 50, the Secret Detection job fails as the clone is not deep enough to contain all of the relevant commits. +If a pipeline is triggered from a Merge Request containing 60 commits while the `GIT_DEPTH` variable +is set to 50 (a [project default](../../../ci/pipelines/settings.md#git-shallow-clone)), +the Secret Detection job fails as the clone is not deep enough to contain all of the +relevant commits. -You can confirm this to be the cause of the error by implementing a [logging level](../../application_security/secret_detection/index.md#logging-level) of `debug`. Once implemented, the logs should look similar to the following example, wherein an "object not found" error can be seen: +To confirm this as the cause of the error, set the +[logging level](../../application_security/secret_detection/index.md#logging-level) to `debug`, then +rerun the pipeline. The logs should look similar to the following example. The text "object not +found" is a symptom of this error. ```plaintext ERRO[2020-11-18T18:05:52Z] object not found @@ -343,7 +356,9 @@ ERRO[2020-11-18T18:05:52Z] object not found [ERRO] [secrets] [2020-11-18T18:05:52Z] ▶ Gitleaks analysis failed: exit status 2 ``` -If this is the case, we can resolve the issue by setting the [`GIT_DEPTH` variable](../../../ci/runners/README.md#shallow-cloning) to a higher value. In order to apply this only to the Secret Detection job, the following can be added to your `.gitlab-ci.yml`: +To resolve the issue, set the [`GIT_DEPTH` CI/CD variable](../../../ci/runners/README.md#shallow-cloning) +to a higher value. To apply this only to the Secret Detection job, the following can be added to +your `.gitlab-ci.yml` file: ```yaml secret_detection: |