Add latest changes from gitlab-org/gitlab@master

3 files changed, 106 insertions, 22 deletions

diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md
index cf058a85e72..a4235afad04 100644
--- a/doc/administration/operations/extra_sidekiq_processes.md
+++ b/doc/administration/operations/extra_sidekiq_processes.md
@@ -70,11 +70,11 @@ To start multiple processes:
    ]
    ```
 
-   `*` cannot be combined with concrete queue names - `*, mailers` will
-   just handle the `mailers` queue.
+   `*` cannot be combined with concrete queue names - `*, mailers`
+   just handles the `mailers` queue.
 
    When `sidekiq-cluster` is only running on a single node, make sure that at least
-   one process is running on all queues using `*`. This means a process will
+   one process is running on all queues using `*`. This means a process is
    This includes queues that have dedicated processes.
 
    If `sidekiq-cluster` is running on more than one node, you can also use
@@ -142,7 +142,7 @@ WARNING:
 Sidekiq cluster is [scheduled](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/240)
 to be the only way to start Sidekiq in GitLab 14.0.
 
-By default, the Sidekiq service will run `sidekiq-cluster`. To disable this behavior,
+By default, the Sidekiq service runs `sidekiq-cluster`. To disable this behavior,
 add the following to the Sidekiq configuration:
 
 ```ruby
@@ -151,7 +151,7 @@ sidekiq['cluster'] = false
 ```
 
 All of the aforementioned configuration options for `sidekiq`
-are available. By default, they will be configured as follows:
+are available. By default, they are configured as follows:
 
 ```ruby
 sidekiq['queue_selector'] = false
@@ -168,14 +168,14 @@ cluster as above.
 
 When disabling `sidekiq_cluster`, you must copy your configuration for
 `sidekiq_cluster`over to `sidekiq`. Anything configured for
-`sidekiq_cluster` will be overridden by the options for `sidekiq` when
+`sidekiq_cluster` is overridden by the options for `sidekiq` when
 setting `sidekiq['cluster'] = true`.
 
-When using this feature, the service called `sidekiq` will now be
+When using this feature, the service called `sidekiq` is now 
 running `sidekiq-cluster`.
 
 The [concurrency](#manage-concurrency) and other options configured
-for Sidekiq will be respected.
+for Sidekiq are respected.
 
 By default, logs for `sidekiq-cluster` go to `/var/log/gitlab/sidekiq`
 like regular Sidekiq logs.
@@ -220,7 +220,7 @@ use all of its resources to perform those operations. To set up a separate
 Each process defined under `sidekiq` starts with a
 number of threads that equals the number of queues, plus one spare thread.
 For example, a process that handles the `process_commit` and `post_receive`
-queues will use three threads in total.
+queues uses three threads in total.
 
 ## Manage concurrency
 
@@ -251,16 +251,16 @@ Running Sidekiq cluster is the default in GitLab 13.0 and later.
    ```
 
 `min_concurrency` and `max_concurrency` are independent; one can be set without
-the other. Setting `min_concurrency` to `0` will disable the limit.
+the other. Setting `min_concurrency` to `0` disables the limit.
 
 For each queue group, let `N` be one more than the number of queues. The
-concurrency factor will be set to:
+concurrency factor are set to:
 
 1. `N`, if it's between `min_concurrency` and `max_concurrency`.
 1. `max_concurrency`, if `N` exceeds this value.
 1. `min_concurrency`, if `N` is less than this value.
 
-If `min_concurrency` is equal to `max_concurrency`, then this value will be used
+If `min_concurrency` is equal to `max_concurrency`, then this value is used
 regardless of the number of queues.
 
 When `min_concurrency` is greater than `max_concurrency`, it is treated as
@@ -287,7 +287,7 @@ Running Sidekiq directly is scheduled to be removed in GitLab
    sudo gitlab-ctl reconfigure
    ```
 
-This will set the concurrency (number of threads) for the Sidekiq process.
+This sets the concurrency (number of threads) for the Sidekiq process.
 
 ## Modify the check interval
 
@@ -353,21 +353,21 @@ you'd use the following:
 
 ### Monitor the `sidekiq-cluster` command
 
-The `sidekiq-cluster` command will not terminate once it has started the desired
-amount of Sidekiq processes. Instead, the process will continue running and
+The `sidekiq-cluster` command does not terminate once it has started the desired
+amount of Sidekiq processes. Instead, the process continues running and
 forward any signals to the child processes. This makes it easy to stop all
 Sidekiq processes as you simply send a signal to the `sidekiq-cluster` process,
 instead of having to send it to the individual processes.
 
 If the `sidekiq-cluster` process crashes or receives a `SIGKILL`, the child
-processes will terminate themselves after a few seconds. This ensures you don't
+processes terminate themselves after a few seconds. This ensures you don't
 end up with zombie Sidekiq processes.
 
 All of this makes monitoring the processes fairly easy. Simply hook up
 `sidekiq-cluster` to your supervisor of choice (for example, runit) and you're good to
 go.
 
-If a child process died the `sidekiq-cluster` command will signal all remaining
+If a child process died the `sidekiq-cluster` command signals all remaining
 process to terminate, then terminate itself. This removes the need for
 `sidekiq-cluster` to re-implement complex process monitoring/restarting code.
 Instead you should make sure your supervisor restarts the `sidekiq-cluster`
@@ -383,7 +383,7 @@ file is written, but this can be changed by passing the `--pidfile` option to
 /opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster --pidfile /var/run/gitlab/sidekiq_cluster.pid process_commit
 ```
 
-Keep in mind that the PID file will contain the PID of the `sidekiq-cluster`
+Keep in mind that the PID file contains the PID of the `sidekiq-cluster`
 command and not the PID(s) of the started Sidekiq processes.
 
 ### Environment
diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md
index abfa65e636d..22487d14e86 100644
--- a/doc/user/application_security/api_fuzzing/index.md
+++ b/doc/user/application_security/api_fuzzing/index.md
@@ -568,6 +568,7 @@ profile increases as the number of tests increases.
 | `FUZZAPI_TARGET_URL`                                        | Base URL of API testing target. |
 | `FUZZAPI_CONFIG`                                            | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/276395) in GitLab 13.12, replaced with default `.gitlab/gitlab-api-fuzzing-config.yml`. API Fuzzing configuration file. |
 |[`FUZZAPI_PROFILE`](#api-fuzzing-profiles)                   | Configuration profile to use during testing. Defaults to `Quick-10`. |
+|[`FUZZAPI_EXCLUDE_PATHS`](#exclude-paths)                    | Exclude API URL paths from testing. |
 |[`FUZZAPI_OPENAPI`](#openapi-specification)                  | OpenAPI Specification file or URL. |
 |[`FUZZAPI_HAR`](#http-archive-har)                           | HTTP Archive (HAR) file. |
 |[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection)          | Postman Collection file. |
@@ -825,6 +826,47 @@ variables:
   FUZZAPI_OVERRIDES_INTERVAL: 300
 ```
 
+### Exclude Paths
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0.
+
+When testing an API it can be useful to exclude certain paths. For example, you might exclude testing of an authentication service or an older version of the API. To exclude paths, use the `FUZZAPI_EXCLUDE_PATHS` CI/CD variable . This variable is specified in your `.gitlab-ci.yml` file. To exclude multiple paths, separate entries using the `;` character. In the provided paths you can use a single character wildcard `?` and `*` for a multiple character wildcard.
+
+To verify the paths are excluded, review the `Tested Operations` and `Excluded Operations` portion of the job output. You should not see any excluded paths listed under `Tested Operations`.
+
+```plaintext
+2021-05-27 21:51:08 [INF] API Security: --[ Tested Operations ]-------------------------
+2021-05-27 21:51:08 [INF] API Security: 201 POST http://target:7777/api/users CREATED
+2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
+2021-05-27 21:51:08 [INF] API Security: --[ Excluded Operations ]-----------------------
+2021-05-27 21:51:08 [INF] API Security: GET http://target:7777/api/messages
+2021-05-27 21:51:08 [INF] API Security: POST http://target:7777/api/messages
+2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
+```
+
+#### Examples of excluding paths
+
+This example excludes the `/auth` resource. This does not exclude child resources (`/auth/child`).
+
+```yaml
+variables:
+  FUZZAPI_EXCLUDE_PATHS=/auth
+```
+
+To exclude `/auth`, and child resources (`/auth/child`), we use a wildcard.
+
+```yaml
+variables:
+  FUZZAPI_EXCLUDE_PATHS=/auth*
+```
+
+To exclude multiple paths we can use the `;` character. In this example we exclude `/auth*` and `/v1/*`.
+
+```yaml
+variables:
+  FUZZAPI_EXCLUDE_PATHS=/auth*;/v1/*
+```
+
 ### Header Fuzzing
 
 Header fuzzing is disabled by default due to the high number of false positives that occur with many
diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md
index e8e5d7a3957..b967b80d85a 100644
--- a/doc/user/application_security/dast_api/index.md
+++ b/doc/user/application_security/dast_api/index.md
@@ -72,8 +72,8 @@ starting in GitLab 14.0, GitLab will not check your repository's root for config
 > Support for OpenAPI Specification using YAML format was
 > [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0.
 
-The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs. 
-This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test. 
+The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs.
+This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test.
 OpenAPI Specifications are provided as a file system resource or URL. Both JSON and YAML OpenAPI formats are supported.
 
 DAST API uses an OpenAPI document to generate the request body. When a request body is required,
@@ -635,6 +635,7 @@ can be added, removed, and modified by creating a custom configuration.
 | `DAST_API_TARGET_URL`                                 | Base URL of API testing target. |
 |[`DAST_API_CONFIG`](#configuration-files)              | DAST API configuration file. Defaults to `.gitlab-dast-api.yml`. |
 |[`DAST_API_PROFILE`](#configuration-files)             | Configuration profile to use during testing. Defaults to `Quick`. |
+|[`FUZZAPI_EXCLUDE_PATHS`](#exclude-paths)              | Exclude API URL paths from testing. |
 |[`DAST_API_OPENAPI`](#openapi-specification)           | OpenAPI specification file or URL. |
 |[`DAST_API_HAR`](#http-archive-har)                    | HTTP Archive (HAR) file. |
 |[`DAST_API_POSTMAN_COLLECTION`](#postman-collection)   | Postman Collection file. |
@@ -894,6 +895,47 @@ variables:
   DAST_API_OVERRIDES_INTERVAL: 300
 ```
 
+### Exclude Paths
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0.
+
+When testing an API it can be useful to exclude certain paths. For example, you might exclude testing of an authentication service or an older version of the API. To exclude paths, use the `FUZZAPI_EXCLUDE_PATHS` CI/CD variable . This variable is specified in your `.gitlab-ci.yml` file. To exclude multiple paths, separate entries using the `;` character. In the provided paths you can use a single character wildcard `?` and `*` for a multiple character wildcard.
+
+To verify the paths are excluded, review the `Tested Operations` and `Excluded Operations` portion of the job output. You should not see any excluded paths listed under `Tested Operations`.
+
+```plaintext
+2021-05-27 21:51:08 [INF] API Security: --[ Tested Operations ]-------------------------
+2021-05-27 21:51:08 [INF] API Security: 201 POST http://target:7777/api/users CREATED
+2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
+2021-05-27 21:51:08 [INF] API Security: --[ Excluded Operations ]-----------------------
+2021-05-27 21:51:08 [INF] API Security: GET http://target:7777/api/messages
+2021-05-27 21:51:08 [INF] API Security: POST http://target:7777/api/messages
+2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
+```
+
+#### Examples
+
+This example excludes the `/auth` resource. This does not exclude child resources (`/auth/child`).
+
+```yaml
+variables:
+  DAST_API_EXCLUDE_PATHS=/auth
+```
+
+To exclude `/auth`, and child resources (`/auth/child`), we use a wildcard.
+
+```yaml
+variables:
+  DAST_API_EXCLUDE_PATHS=/auth*
+```
+
+To exclude multiple paths we use the `;` character. In this example we exclude `/auth*` and `/v1/*`.
+
+```yaml
+variables:
+  DAST_API_EXCLUDE_PATHS=/auth*;/v1/*
+```
+
 ## Running your first scan
 
 When configured correctly, a CI/CD pipeline contains a `dast` stage and an `dast_api` job. The job only fails when an invalid configuration is provided. During normal operation, the job always succeeds even if vulnerabilities are identified during testing.
@@ -1089,7 +1131,7 @@ The best-suited solution will depend on whether or not your target API changes f
 
 #### Static environment solution
 
-This solution is for pipelines in which the target API URL doesn't change (is static). 
+This solution is for pipelines in which the target API URL doesn't change (is static).
 
 **Add environmental variable**
 
@@ -1106,7 +1148,7 @@ include:
 
 #### Dynamic environment solutions
 
-In a dynamic environment your target API changes for each different deployment. In this case, there is more than one possible solution, we recommend you use the `environment_url.txt` file when dealing with dynamic environments. 
+In a dynamic environment your target API changes for each different deployment. In this case, there is more than one possible solution, we recommend you use the `environment_url.txt` file when dealing with dynamic environments.
 
 **Use environment_url.txt**