1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
|
---
stage: Secure
group: Fuzz Testing
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: reference, howto
---
# Coverage Guided Fuzz Testing **(ULTIMATE)**
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3226) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha).
GitLab allows you to add coverage-guided fuzz testing to your pipelines. This helps you discover
bugs and potential security issues that other QA processes may miss. Coverage-guided fuzzing sends
random inputs to an instrumented version of your application in an effort to cause unexpected
behavior, such as a crash. Such behavior indicates a bug that you should address.
We recommend that you use fuzz testing in addition to the other security scanners in [GitLab Secure](../index.md)
and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md),
you can run your coverage guided fuzz tests as part your CI/CD workflow. You can take advantage of
Coverage Guided Fuzzing by including the CI job in your existing `.gitlab-ci.yml` file.
## Supported fuzzing engines and languages
GitLab supports these languages through the fuzzing engine listed for each. We currently provide a Docker image for apps written in Go, but you can test the other languages below by providing a Docker image with the fuzz engine to run your app.
| Language | Fuzzing Engine | Example |
|----------|---------------------------------------------------------------------------|---------|
| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | |
| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | |
| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | |
## Configuration
To enable fuzzing, you must
[include](../../../ci/yaml/README.md#includetemplate)
the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml)
provided as part of your GitLab installation.
To do so, add the following to your `.gitlab-ci.yml` file:
```yaml
include:
- template: Coverage-Fuzzing.gitlab-ci.yml
```
The included template makes available the [hidden job](../../../ci/yaml/README.md#hide-jobs)
`.fuzz_base`, which you must [extend](../../../ci/yaml/README.md#extends) for each of your fuzz
targets. Each fuzz target **must** have a separate job. For example, the
[go-fuzzing-example project](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example)
contains one job that extends `.fuzz_base` for its single fuzz target.
The `my_fuzz_target` job (the separate job for your fuzz target) does the following:
- Extends `.fuzz_base`.
- Compiles the fuzz target with [go-fuzz](https://github.com/dvyukov/go-fuzz).
- Runs the target with the `gitlab-cov-fuzz` command, which is available to each job that extends
`.fuzz_base`.
- Runs on a fuzz stage that usually comes after a test stage.
The `gitlab-cov-fuzz` is a command-line tool that runs the instrumented application. It parses and
analyzes the exception information that the fuzzer outputs. It also downloads the [corpus](#glossary)
and crash events from previous pipelines automatically. This helps your fuzz targets build on the progress of
previous fuzzing jobs. The parsed crash events and data are written to
`gl-coverage-fuzzing-report.json`.
### Artifacts
Each fuzzing step outputs these artifacts:
- `gl-coverage-fuzzing-report.json`: This file's format may change in future releases.
- `artifacts.zip`: This file contains two directories:
- `corpus`: Holds all test cases generated by the current and all previous jobs.
- `crashes`: Holds all crash events the current job encountered as well as those not fixed in
previous jobs.
### Types of Fuzzing Jobs
There are two types of jobs:
- Fuzzing: Standard fuzzing session. You can configure a long session through a user defined
timeout.
- Regression: Run the fuzz targets through the accumulated test cases generated by previous fuzzing
sessions plus fixed crashes from previous sessions. This is usually very quick.
Here's our current suggestion for configuring your fuzz target's timeout:
- Set `COVERAGE_FUZZING_BRANCH` to the branch where you want to run long-running (async) fuzzing
jobs. This is `master` by default.
- Use regression or short-running fuzzing jobs for other branches or merge requests.
This suggestion helps find new bugs on the development branch and catch old bugs in merge requests
(like unit tests).
You can configure this by passing `--regression=false/true` to `gitlab-cov-fuzz` as the [Go example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example/-/blob/master/.gitlab-ci.yml)
shows. Also note that `gitlab-cov-fuzz` is a wrapper, so you can pass those arguments to configure
any option available in the underlying fuzzing engine.
### Available variables
| Environment variable | Description |
|---------------------------|--------------------------------------------------------------------|
| `COVERAGE_FUZZING_BRANCH` | The branch for long-running fuzzing jobs. The default is `master`. |
### Additional Configuration
The `gitlab-cov-fuzz` command passes all arguments it receives to the underlying fuzzing engine. You
can therefore use all the options available in that fuzzing engine. For more information on these
options, see the underlying fuzzing engine's documentation.
### Glossary
- Seed corpus: The set of test cases given as initial input to the fuzz target. This usually speeds
up the fuzz target substantially. This can be either manually created test cases or auto-generated
with the fuzz target itself from previous runs.
- Corpus: The set of meaningful test cases that are generated while the fuzzer is running. Each
meaningful test case produces new coverage in the tested program. It's advised to re-use the
corpus and pass it to subsequent runs.
|
