diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-05-20 09:10:43 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-05-20 09:10:43 +0300 |
commit | 763dd8a47a4e353119f54de0919e10a4fa1de879 (patch) | |
tree | 20f4307a35accd0458165a516bf2af3ce1b2664e /doc | |
parent | 552cf9f9c4e086a57585e2fa312ae2bbb88d9ca4 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/install/next_steps.md | 45 | ||||
-rw-r--r-- | doc/user/application_security/dast/browser_based.md | 215 | ||||
-rw-r--r-- | doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png | bin | 0 -> 203980 bytes | |||
-rw-r--r-- | doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png | bin | 0 -> 155046 bytes | |||
-rw-r--r-- | doc/user/application_security/dast/img/dast_auth_report.jpg | bin | 0 -> 121121 bytes |
5 files changed, 223 insertions, 37 deletions
diff --git a/doc/install/next_steps.md b/doc/install/next_steps.md index 3e73da123fb..0a2a1c250d6 100644 --- a/doc/install/next_steps.md +++ b/doc/install/next_steps.md @@ -9,11 +9,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w Here are a few resources you might want to check out after completing the installation. -## License +## Email and notifications -- [Upload a license](../user/admin_area/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): - Activate all GitLab Enterprise Edition functionality with a license. -- [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. +- [SMTP](https://docs.gitlab.com/omnibus/settings/smtp.html): Configure SMTP + for proper email notifications support. + +## CI/CD + +- [Set up runners](https://docs.gitlab.com/runner/): Set up one or more GitLab + Runners, the agents that are responsible for all of the GitLab CI/CD features. +- [GitLab Pages](../administration/pages/index.md): Configure GitLab Pages to + allow hosting of static sites. +- [GitLab Registry](../administration/packages/container_registry.md): With the + GitLab Container Registry, every project can have its own space to store Docker + images. ## Security @@ -24,11 +33,7 @@ installation. - [LDAP](../administration/auth/ldap/index.md): Configure LDAP to be used as an authentication mechanism for GitLab. - -## Email and notifications - -- [SMTP](https://docs.gitlab.com/omnibus/settings/smtp.html): Configure SMTP - for proper email notifications support. +- [SAML and OAuth](../integration/omniauth.md): Authenticate via online services like Okta, Google, Azure AD, and more. ## Backup and upgrade @@ -40,15 +45,16 @@ installation. policies governing version naming, as well as release pace for major, minor, patch, and security releases. -## CI/CD +## License -- [Set up runners](https://docs.gitlab.com/runner/): Set up one or more GitLab - Runners, the agents that are responsible for all of the GitLab CI/CD features. -- [GitLab Pages](../administration/pages/index.md): Configure GitLab Pages to - allow hosting of static sites. -- [GitLab Registry](../administration/packages/container_registry.md): With the - GitLab Container Registry, every project can have its own space to store Docker - images. +- [Upload a license](../user/admin_area/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): + Activate all GitLab Enterprise Edition functionality with a license. +- [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. + +## Cross-repo Code Search + +- [Advanced Search](../integration/elasticsearch.md): Leverage Elasticsearch for + faster, more advanced code search across your entire GitLab instance. ## Scaling and replication @@ -56,8 +62,3 @@ installation. GitLab supports several different types of clustering. - [Geo replication](../administration/geo/index.md): Geo is the solution for widely distributed development teams. - -## Search - -- [Advanced Search](../integration/elasticsearch.md): Leverage Elasticsearch for - faster, more advanced code search across your entire GitLab instance. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index b8c3529225c..aaf4496076f 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -38,9 +38,10 @@ An example configuration might look like the following: include: - template: DAST.gitlab-ci.yml -variables: - DAST_WEBSITE: "https://example.com" - DAST_BROWSER_SCAN: "true" +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" ``` ### Available variables @@ -64,8 +65,12 @@ The browser-based crawler can be configured using CI/CD variables. | `DAST_PASSWORD` | string | `p@55w0rd` | The password to enter into the password field on the sign-in HTML form. | | `DAST_USERNAME_FIELD` | selector | `id:user` | A selector describing the username field on the sign-in HTML form. | | `DAST_PASSWORD_FIELD` | selector | `css:.password-field` | A selector describing the password field on the sign-in HTML form. | -| `DAST_SUBMIT_FIELD` | selector | `xpath://input[@value='Login']` | A selector describing the element that when clicked submits the login form or the password form of a multi-page login process. | +| `DAST_SUBMIT_FIELD` | selector | `xpath://input[@value='Login']` | A selector describing the element that when clicked submits the login form, or the password form of a multi-page login process. | | `DAST_FIRST_SUBMIT_FIELD` | selector | `.submit` | A selector describing the element that when clicked submits the username form of a multi-page login process. | +| `DAST_BROWSER_AUTH_REPORT` | boolean | `true` | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | +| `DAST_AUTH_VERIFICATION_URL` | URL | `http://site.com/welcome` | Verifies successful authentication by checking the URL once the login form has been submitted. | +| `DAST_BROWSER_AUTH_VERIFICATION_SELECTOR` | selector | `css:.user-photo` | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. | +| `DAST_BROWSER_AUTH_VERIFICATION_LOGIN_FORM` | boolean | `true` | Verifies successful authentication by checking for the lack of a login form once the login form has been submitted. | The [DAST variables](index.md#available-variables) `SECURE_ANALYZERS_PREFIX`, `DAST_FULL_SCAN_ENABLED`, `DAST_AUTO_UPDATE_ADDONS`, `DAST_EXCLUDE_RULES`, `DAST_REQUEST_HEADERS`, `DAST_HTML_REPORT`, `DAST_MARKDOWN_REPORT`, `DAST_XML_REPORT`, `DAST_INCLUDE_ALPHA_VULNERABILITIES`, `DAST_PATHS_FILE`, `DAST_PATHS`, `DAST_ZAP_CLI_OPTIONS`, and `DAST_ZAP_LOG_CONFIGURATION` are also compatible with browser-based crawler scans. @@ -75,13 +80,47 @@ The [DAST variables](index.md#available-variables) `SECURE_ANALYZERS_PREFIX`, `D Selectors are used by CI/CD variables to specify the location of an element displayed on a page in a browser. Selectors have the format `type`:`search string`. The crawler will search for the selector using the search string based on the type. -| Selector type | Example | Description | -| ------------- | ------------------------------ | ----------- | -| `css` | `css:.password-field` | Searches for a HTML element having the supplied CSS selector. Selectors should be as specific as possible for performance reasons. | -| `id` | `id:element` | Searches for an HTML element with the provided element ID. | -| `name` | `name:element` | Searches for an HTML element with the provided element name. | -| `xpath` | `xpath://*[@id="my-button"]/a` | Searches for a HTML element with the provided XPath. Note that XPath searches are expected to be less performant than other searches. | -| None provided | `a.click-me` | Defaults to searching using a CSS selector. | +| Selector type | Example | Description | +| ------------- | ---------------------------------- | ----------- | +| `css` | `css:.password-field` | Searches for a HTML element having the supplied CSS selector. Selectors should be as specific as possible for performance reasons. | +| `id` | `id:element` | Searches for an HTML element with the provided element ID. | +| `name` | `name:element` | Searches for an HTML element with the provided element name. | +| `xpath` | `xpath://input[@id="my-button"]/a` | Searches for a HTML element with the provided XPath. Note that XPath searches are expected to be less performant than other searches. | +| None provided | `a.click-me` | Defaults to searching using a CSS selector. | + +##### Find selectors with Google Chrome + +Chrome DevTools element selector tool is an effective way to find a selector. + +1. Open Chrome and navigate to the page where you would like to find a selector, for example, the login page for your site. +1. Open the `Elements` tab in Chrome DevTools with the keyboard shortcut `Command + Shift + c` in macOS or `Ctrl + Shift + c` in Windows. +1. Select the `Select an element in the page to select it` tool. + ![search-elements](img/dast_auth_browser_scan_search_elements.png) +1. Select the field on your page that you would like to know the selector for. +1. Once the tool is active, highlight a field you wish to view the details of. + ![highlight](img/dast_auth_browser_scan_highlight.png) +1. Once highlighted, you can see the element's details, including attributes that would make a good candidate for a selector. + +In this example, the `id="user_login"` appears to be a good candidate. You can use this as a selector as the DAST username field by setting `DAST_USERNAME_FIELD: "css:[id=user_login]"`, or more simply, `DAST_USERNAME_FIELD: "id:user_login"`. + +##### Choose the right selector + +Judicious choice of selector leads to a scan that is resilient to the application changing. + +In order of preference, it is recommended to choose as selectors: + +- `id` fields. These are generally unique on a page, and rarely change. +- `name` fields. These are generally unique on a page, and rarely change. +- `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field. +- Presence of field specific data attributes, such as the selector, `"css:[data-username]"` when the `data-username` field has any value on the username field. +- Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`. + +When using selectors to locate specific fields we recommend you avoid searching on: + +- Any `id`, `name`, `attribute`, `class` or `value` that is dynamically generated. +- Generic class names, such as `column-10` and `dark-grey`. +- XPath searches as they are less performant than other selector searches. +- Unscoped searches, such as those beginning with `css:*` and `xpath://*`. ## Vulnerability detection @@ -93,6 +132,151 @@ When running a full scan, active vulnerability checks executed by DAST/ZAP do no For example, for a target website that contains forms with Anti-CSRF tokens, a passive scan will scan as intended because the browser displays pages/forms as if a user is viewing the page. However, active vulnerability checks run in a full scan will not be able to submit forms containing Anti-CSRF tokens. In such cases we recommend you disable Anti-CSRF tokens when running a full scan. +## Authentication + +If your application hosts content only available to logged in users then you will likely get much higher crawl coverage of your application by configuring authentication. +We recommended you periodically confirm that authentication is still working as this tends to break over +time due to changes to the application. + +Authentication supports single form logins, multi-step login forms, and authenticating to URLs outside of the configured target URL. + +An example configuration that authenticates a user might look like the following: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" + DAST_AUTH_URL: "https://login.example.com/" + DAST_USERNAME: "admin" + DAST_PASSWORD: "P@55w0rd!" + DAST_USERNAME_FIELD: "name:username" + DAST_PASSWORD_FIELD: "name:password" + DAST_SUBMIT_FIELD: "css:button[type='submit']" +``` + +### Log in using automatic detection of the login form + +By providing a `DAST_USERNAME`, `DAST_PASSWORD`, and `DAST_AUTH_URL`, the browser-based crawler will attempt to authenticate to the +target application by locating the login form based on a determination about whether or not the form contains username or password fields. + +Automatic detection is "best-effort", and depending on the application being scanned may provide either a resilient login experience or one that fails to authenticate the user. + +Login process: + +1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located. + 1. If a form contains a username and password field, `DAST_USERNAME` and `DAST_PASSWORD` is inputted into the respective fields, the form submit button is clicked and the user is logged in. + 1. If a form contains only a username field, it is assumed that the login form is multi-step. + 1. The `DAST_USERNAME` is inputted into the username field and the form submit button is clicked. + 1. The subsequent pages loads where it is expected that a form exists and contains a password field. If found, `DAST_PASSWORD` is inputted, form submit button is clicked and the user is logged in. + +### Log in using explicit selection of the login form + +By providing a `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD`, in addition to the fields required for automatic login, +the browser-based crawler will attempt to authenticate to the target application by locating the login form based on the selectors provided. +Most applications will benefit from this approach to authentication. + +Login process: + +1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located. + 1. If the `DAST_FIRST_SUBMIT_FIELD` is not defined, then `DAST_USERNAME` is inputted into `DAST_USERNAME_FIELD`, `DAST_PASSWORD` is inputted into `DAST_PASSWORD_FIELD`, `DAST_SUBMIT_FIELD` is clicked and the user is logged in. + 1. If the `DAST_FIRST_SUBMIT_FIELD` is defined, then it is assumed that the login form is multi-step. + 1. The `DAST_USERNAME` is inputted into the `DAST_USERNAME_FIELD` field and the `DAST_FIRST_SUBMIT_FIELD` is clicked. + 1. The subsequent pages loads where the `DAST_PASSWORD` is inputted into the `DAST_PASSWORD_FIELD` field, the `DAST_SUBMIT_FIELD` is clicked and the user is logged in. + +### Verifying successful login + +Once the login form has been submitted, the browser-based crawler determines if the login was successful. Unsuccessful attempts at authentication cause the scan to halt. + +Following the submission of the login form, authentication is determined to be unsuccessful when: + +- A `400` or `500` series HTTP response status code is returned. +- A new cookie/browser storage value determined to be sufficiently random has not been set. + +In addition to these checks, the user can configure their own verification checks. +Each of the following checks can be used in conjunction with one another, if none are configured by default the presence of a login form is checked. + +#### Verifying based on the URL + +When `DAST_AUTH_VERIFICATION_URL` is configured, the URL displayed in the browser tab post login form submission is directly compared to the URL in the CI/CD variable. +If these are not exactly the same, authentication is deemed to be unsuccessful. + +For example: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" + ... + DAST_AUTH_VERIFICATION_URL: "https://example.com/user/welcome" +``` + +#### Verify based on presence of an element + +When `DAST_BROWSER_AUTH_VERIFICATION_SELECTOR` is configured, the page displayed in the browser tab is searched for an element described by the selector in the CI/CD variable. +If no element is found, authentication is deemed to be unsuccessful. + +For example: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" + ... + DAST_BROWSER_AUTH_VERIFICATION_SELECTOR: "css:.welcome-user" +``` + +#### Verify based on presence of a login form + +When `DAST_BROWSER_AUTH_VERIFICATION_LOGIN_FORM` is configured, the page displayed in the browser tab is searched for a form that is detected to be a login form. +If any such form is found, authentication is deemed to be unsuccessful. + +For example: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" + ... + DAST_BROWSER_AUTH_VERIFICATION_LOGIN_FORM: "true" +``` + +### Configure the authentication debug output + +It is often difficult to understand the cause of an authentication failure when running DAST in a CI/CD pipeline. +To assist users in debugging authentication issues, a debug report can be generated and saved as a job artifact. +This HTML report contains all steps the browser crawler took, along with HTTP requests and responses, the Document Object Model (DOM) and screenshots. + +![dast-auth-report](img/dast_auth_report.jpg) + +An example configuration where the authentication debug report is exported may look like the following: + +```yaml +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" + ... + DAST_BROWSER_AUTH_REPORT: "true" + artifacts: + paths: [gl-dast-debug-auth-report.html] +``` + ## Managing scan time It is expected that running the browser-based crawler will result in better coverage for many web applications, when compared to the normal GitLab DAST solution. @@ -116,10 +300,11 @@ For example, the following job definition enables the browsing module and the au include: - template: DAST.gitlab-ci.yml -variables: - DAST_WEBSITE: "https://my.site.com" - DAST_BROWSER_SCAN: "true" - DAST_BROWSER_LOG: "brows:debug,auth:debug" +dast: + variables: + DAST_WEBSITE: "https://my.site.com" + DAST_BROWSER_SCAN: "true" + DAST_BROWSER_LOG: "brows:debug,auth:debug" ``` ### Log message format diff --git a/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png b/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png Binary files differnew file mode 100644 index 00000000000..132c9f9c991 --- /dev/null +++ b/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png diff --git a/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png b/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png Binary files differnew file mode 100644 index 00000000000..4e1dca626bc --- /dev/null +++ b/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png diff --git a/doc/user/application_security/dast/img/dast_auth_report.jpg b/doc/user/application_security/dast/img/dast_auth_report.jpg Binary files differnew file mode 100644 index 00000000000..5d9d98045ef --- /dev/null +++ b/doc/user/application_security/dast/img/dast_auth_report.jpg |