diff options
Diffstat (limited to 'doc/user/application_security/container_scanning/index.md')
| -rw-r--r-- | doc/user/application_security/container_scanning/index.md | 217 |
1 files changed, 81 insertions, 136 deletions
diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 6e52d7dbdcf..0ffe83cdfc9 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Defend +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Container Scanning **(ULTIMATE)** @@ -8,35 +11,28 @@ type: reference, howto ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can check your Docker -images (or more precisely the containers) for known vulnerabilities by using -[Clair](https://github.com/quay/clair) and [klar](https://github.com/optiopay/klar), -two open source tools for Vulnerability Static Analysis for containers. +Your application's Docker image may itself be based on Docker images that contain known +vulnerabilities. By including an extra job in your pipeline that scans for those vulnerabilities and +displays them in a merge request, you can use GitLab to audit your Docker-based apps. +By default, container scanning in GitLab is based on [Clair](https://github.com/quay/clair) and +[Klar](https://github.com/optiopay/klar), which are open-source tools for vulnerability static analysis in +containers. [GitLab's Klar analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar/) +scans the containers and serves as a wrapper for Clair. -You can take advantage of Container Scanning by either [including the CI job](#configuration) in -your existing `.gitlab-ci.yml` file or by implicitly using -[Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). - -GitLab checks the Container Scanning report, compares the found vulnerabilities -between the source and target branches, and shows the information right on the -merge request. - - - -## Contribute your scanner +NOTE: **Note:** +To integrate security scanners other than Clair and Klar into GitLab, see +[Security scanner integration](../../../development/integrations/secure.md). -The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate other security scanners into GitLab. +You can enable container scanning by doing one of the following: -## Use cases +- [Include the CI job](#configuration) in your existing `.gitlab-ci.yml` file. +- Implicitly use [Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning-ultimate) + provided by [Auto DevOps](../../../topics/autodevops/index.md). -If you distribute your application with Docker, then there's a great chance -that your image is based on other Docker images that may in turn contain some -known vulnerabilities that could be exploited. +GitLab compares the found vulnerabilities between the source and target branches, and shows the +information directly in the merge request. -Having an extra job in your pipeline that checks for those vulnerabilities, -and the fact that they are displayed inside a merge request, makes it very easy -to perform audits for your Docker-based apps. + <!-- NOTE: The container scanning tool references the following heading in the code, so if you make a change to this heading, make sure to update the documentation URLs used in the @@ -44,33 +40,28 @@ to perform audits for your Docker-based apps. ## Requirements -To enable Container Scanning in your pipeline, you need: - -- A GitLab Runner with the - [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or - [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) - executor. -- Docker `18.09.03` or higher installed on the machine where the Runners are - running. If you're using the shared Runners on GitLab.com, this is already - the case. -- To [build and push](../../packages/container_registry/index.md#container-registry-examples-with-gitlab-cicd) - your Docker image to your project's Container Registry. - The name of the Docker image should use the following - [predefined environment variables](../../../ci/variables/predefined_variables.md) - as defined below: +To enable Container Scanning in your pipeline, you need the following: + +- [GitLab Runner](https://docs.gitlab.com/runner/) with the [Docker](https://docs.gitlab.com/runner/executors/docker.html) + or [Kubernetes](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +- Docker `18.09.03` or higher installed on the same computer as the Runner. If you're using the + shared Runners on GitLab.com, then this is already the case. +- [Build and push](../../packages/container_registry/index.md#container-registry-examples-with-gitlab-cicd) + your Docker image to your project's container registry. The name of the Docker image should use + the following [predefined environment variables](../../../ci/variables/predefined_variables.md): ```plaintext $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA ``` - These can be used directly in your `.gitlab-ci.yml` file: + You can use these directly in your `.gitlab-ci.yml` file: ```yaml build: - image: docker:19.03.8 + image: docker:19.03.11 stage: build services: - - docker:19.03.8-dind + - docker:19.03.11-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: @@ -81,45 +72,40 @@ To enable Container Scanning in your pipeline, you need: ## Configuration -For GitLab 11.9 and later, to enable Container Scanning, you must -[include](../../../ci/yaml/README.md#includetemplate) the -[`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) -that's provided as a part of your GitLab installation. -For GitLab versions earlier than 11.9, you can copy and use the job as defined -in that template. +How you enable Container Scanning depends on your GitLab version: -Add the following to your `.gitlab-ci.yml` file: +- GitLab 11.9 and later: [Include](../../../ci/yaml/README.md#includetemplate) the + [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) + that comes with your GitLab installation. +- GitLab versions earlier than 11.9: Copy and use the job from the + [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml). + +To include the `Container-Scanning.gitlab-ci.yml` template (GitLab 11.9 and later), add the +following to your `.gitlab-ci.yml` file: ```yaml include: - template: Container-Scanning.gitlab-ci.yml ``` -The included template will: +The included template: -1. Create a `container_scanning` job in your CI/CD pipeline. -1. Pull the already built Docker image from your project's - [Container Registry](../../packages/container_registry/index.md) (see [requirements](#requirements)) - and scan it for possible vulnerabilities. +- Creates a `container_scanning` job in your CI/CD pipeline. +- Pulls the built Docker image from your project's [Container Registry](../../packages/container_registry/index.md) + (see [requirements](#requirements)) and scans it for possible vulnerabilities. -The results will be saved as a +GitLab saves the results as a [Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) -that you can later download and analyze. -Due to implementation limitations, we always take the latest Container Scanning -artifact available. Behind the scenes, the -[GitLab Klar analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar/) -is used and runs the scans. +that you can download and analyze later. When downloading, you always receive the most-recent +artifact. -The following is a sample `.gitlab-ci.yml` that will build your Docker image, -push it to the Container Registry, and run Container Scanning: +The following is a sample `.gitlab-ci.yml` that builds your Docker image, pushes it to the Container +Registry, and scans the containers: ```yaml variables: DOCKER_DRIVER: overlay2 -services: - - docker:19.03.8-dind - stages: - build - test @@ -127,6 +113,8 @@ stages: build: image: docker:stable stage: build + services: + - docker:19.03.11-dind variables: IMAGE: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: @@ -141,11 +129,15 @@ include: ### Customizing the Container Scanning settings -You can change container scanning settings by using the [`variables`](../../../ci/yaml/README.md#variables) -parameter in your `.gitlab-ci.yml` to change [environment variables](#available-variables). +There may be cases where you want to customize how GitLab scans your containers. For example, you +may want to enable more verbose output from Clair or Klar, access a Docker registry that requires +authentication, and more. To change such settings, use the [`variables`](../../../ci/yaml/README.md#variables) +parameter in your `.gitlab-ci.yml` to set [environment variables](#available-variables). +The environment variables you set in your `.gitlab-ci.yml` overwrite those in +`Container-Scanning.gitlab-ci.yml`. -In the following example, we [include](../../../ci/yaml/README.md#include) the template and also -set the `CLAIR_OUTPUT` variable to `High`: +This example [includes](../../../ci/yaml/README.md#include) the Container Scanning template and +enables verbose output from Clair by setting the `CLAIR_OUTPUT` environment variable to `High`: ```yaml include: @@ -155,9 +147,6 @@ variables: CLAIR_OUTPUT: High ``` -The `CLAIR_OUTPUT` variable defined in the main `gitlab-ci.yml` will overwrite what's -defined in `Container-Scanning.gitlab-ci.yml`, changing the Container Scanning behavior. - <!-- NOTE: The container scanning tool references the following heading in the code, so if you" make a change to this heading, make sure to update the documentation URLs used in the" container scanning tool (https://gitlab.com/gitlab-org/security-products/analyzers/klar)" --> @@ -188,13 +177,9 @@ using environment variables. ### Overriding the Container Scanning template -CAUTION: **Deprecation:** -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. - -If you want to override the job definition (for example, change properties like -`variables`), you need to declare a `container_scanning` job after the -template inclusion and specify any additional keys under it. For example: +If you want to override the job definition (for example, to change properties like `variables`), you +must declare a `container_scanning` job after the template inclusion, and then +specify any additional keys. For example: ```yaml include: @@ -205,15 +190,20 @@ container_scanning: GIT_STRATEGY: fetch ``` +CAUTION: **Deprecated:** +GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic). +When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) +instead. + ### Vulnerability whitelisting -If you want to whitelist specific vulnerabilities, you'll need to: +To whitelist specific vulnerabilities, follow these steps: -1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions described in the - [overriding the Container Scanning template](#overriding-the-container-scanning-template) section of this document. -1. Define the whitelisted vulnerabilities in a YAML file named `clair-whitelist.yml` which must use the format described - in the [whitelist example file](https://github.com/arminc/clair-scanner/blob/v12/example-whitelist.yaml). -1. Add the `clair-whitelist.yml` file to the Git repository of your project. +1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in + [overriding the Container Scanning template](#overriding-the-container-scanning-template). +1. Define the whitelisted vulnerabilities in a YAML file named `clair-whitelist.yml`. This must use + the format described in the [whitelist example file](https://github.com/arminc/clair-scanner/blob/v12/example-whitelist.yaml). +1. Add the `clair-whitelist.yml` file to your project's Git repository. ### Running Container Scanning in an offline environment @@ -286,14 +276,13 @@ this with a pipeline means you won't have to do it manually each time. You can u ```yaml image: docker:stable -services: - - docker:19.03.8-dind - stages: - build build_latest_vulnerabilities: stage: build + services: + - docker:19.03.11-dind script: - docker pull arminc/clair-db:latest - docker tag arminc/clair-db:latest $CI_REGISTRY/namespace/clair-vulnerabilities-db @@ -339,11 +328,10 @@ The results are stored in `gl-container-scanning-report.json`. ## Reports JSON format -CAUTION: **Caution:** -The JSON report artifacts are not a public API of Container Scanning and their format may change in the future. +The Container Scanning tool emits a JSON report file. For more information, see the +[schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/container-scanning-report-format.json). -The Container Scanning tool emits a JSON report file. Here is an example of the report structure with all important parts of -it highlighted: +Here's an example Container Scanning report: ```json-doc { @@ -400,49 +388,6 @@ it highlighted: } ``` -CAUTION: **Deprecation:** -Beginning with GitLab 12.9, container scanning no longer reports `undefined` severity and confidence levels. - -Here is the description of the report file structure nodes and their meaning. All fields are mandatory to be present in -the report JSON unless stated otherwise. Presence of optional fields depends on the underlying analyzers being used. - -| Report JSON node | Description | -|------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `version` | Report syntax version used to generate this JSON. | -| `vulnerabilities` | Array of vulnerability objects. | -| `vulnerabilities[].id` | Unique identifier of the vulnerability. | -| `vulnerabilities[].category` | Where this vulnerability belongs (for example, SAST or Container Scanning). For Container Scanning, it will always be `container_scanning`. | -| `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | -| `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | -| `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | -| `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | -| `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | ID of the scanner as a snake_case string. | -| `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | -| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | -| `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | -| `vulnerabilities[].location.dependency.package` | A node that provides the information on the package where the vulnerability is located. | -| `vulnerabilities[].location.dependency.package.name` | Name of the package where the vulnerability is located. | -| `vulnerabilities[].location.dependency.version` | Version of the vulnerable package. Optional. | -| `vulnerabilities[].location.operating_system` | The operating system that contains the vulnerable package. | -| `vulnerabilities[].location.image` | The Docker image that was analyzed. | -| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | -| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`). | -| `vulnerabilities[].identifiers[].name` | Name of the identifier for display purpose. | -| `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purpose. | -| `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | -| `vulnerabilities[].links` | An array of references to external documentation pieces or articles that describe the vulnerability further. Optional. | -| `vulnerabilities[].links[].name` | Name of the vulnerability details link. Optional. | -| `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | -| `remediations` | An array of objects containing information on cured vulnerabilities along with patch diffs to apply. Empty if no remediations provided by an underlying analyzer. | -| `remediations[].fixes` | An array of strings that represent references to vulnerabilities fixed by this particular remediation. | -| `remediations[].fixes[].id` | The ID of a fixed vulnerability. | -| `remediations[].fixes[].cve` | (**DEPRECATED - use `remediations[].fixes[].id` instead**) A string value that describes a fixed vulnerability in the same format as `vulnerabilities[].cve`. | -| `remediations[].summary` | Overview of how the vulnerabilities have been fixed. | -| `remediations[].diff` | base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | - ## Security Dashboard The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all @@ -473,7 +418,7 @@ Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vu ## Troubleshooting -### docker: Error response from daemon: failed to copy xattrs +### `docker: Error response from daemon: failed to copy xattrs` When the GitLab Runner uses the Docker executor and NFS is used (for example, `/var/lib/docker` is on an NFS mount), Container Scanning might fail with @@ -486,4 +431,4 @@ docker: Error response from daemon: failed to copy xattrs: failed to set xattr " This is a result of a bug in Docker which is now [fixed](https://github.com/containerd/continuity/pull/138 "fs: add WithAllowXAttrErrors CopyOpt"). To prevent the error, ensure the Docker version that the Runner is using is `18.09.03` or higher. For more information, see -[issue #10241](https://gitlab.com/gitlab-org/gitlab/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). +[issue #10241](https://gitlab.com/gitlab-org/gitlab/-/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). |