diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-18 21:16:08 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-18 21:16:08 +0300 |
commit | 5ba99858f15c33bf96f94cc5e9663f01c3532689 (patch) | |
tree | d9a4f77242ff797c6cdbf8a12f6644d347cf8935 /doc | |
parent | 9ee9d3012b3747270beb6dec6200d632d8afd0e3 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/reference_architectures/index.md | 2 | ||||
-rw-r--r-- | doc/api/integrations.md | 4 | ||||
-rw-r--r-- | doc/api/job_artifacts.md | 2 | ||||
-rw-r--r-- | doc/api/users.md | 4 | ||||
-rw-r--r-- | doc/ci/components/index.md | 11 | ||||
-rw-r--r-- | doc/ci/jobs/job_artifacts.md | 20 | ||||
-rw-r--r-- | doc/development/architecture.md | 1 | ||||
-rw-r--r-- | doc/development/internal_analytics/service_ping/index.md | 2 | ||||
-rw-r--r-- | doc/devsecops.md | 10 | ||||
-rw-r--r-- | doc/integration/jira/troubleshooting.md | 69 | ||||
-rw-r--r-- | doc/operations/feature_flags.md | 2 | ||||
-rw-r--r-- | doc/user/application_security/dast/authentication.md | 189 | ||||
-rw-r--r-- | doc/user/application_security/dast/authentication_troubleshooting.md | 196 | ||||
-rw-r--r-- | doc/user/application_security/policies/index.md | 2 | ||||
-rw-r--r-- | doc/user/packages/npm_registry/index.md | 20 | ||||
-rw-r--r-- | doc/user/project/repository/code_suggestions/self_managed.md | 3 |
16 files changed, 280 insertions, 257 deletions
diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index d6fdcbf7e04..c4827695716 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -18,7 +18,7 @@ the _total_ load that comes with such a user count based on real data along with However, it should be noted that in some cases, known heavy scenarios such as [large monorepos](#large-monorepos) or notable [additional workloads](#additional-workloads) may require adjustments to be made. -For each Reference Architecture, the details of what they have been tested against can be found respectively in the "Testing Methodology" section of each page. +For details about what each Reference Architecture has been tested against, see the "Testing Methodology" section of each page. ### GitLab package (Omnibus) diff --git a/doc/api/integrations.md b/doc/api/integrations.md index a1f504a3fd8..f713d845762 100644 --- a/doc/api/integrations.md +++ b/doc/api/integrations.md @@ -128,8 +128,8 @@ Parameters: | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `api_key` | string | true | User API token. User must have access to task. All comments are attributed to this user. | -| `restrict_to_branch` | string | false | Comma-separated list of branches to be are automatically inspected. Leave blank to include all branches. | +| `api_key` | string | true | User API token. The user must have access to the task. All comments are attributed to this user. | +| `restrict_to_branch` | string | false | Comma-separated list of branches to be automatically inspected. Leave blank to include all branches. | ### Disable Asana diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index 8fe97a2fab6..31a72c44d02 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -184,7 +184,7 @@ Possible response status codes: Download a single artifact file for a specific job of the latest **successful** pipeline for the given reference name from inside the job's artifacts archive. -The file is extracted from the archive and streamed to the client. +The file is extracted from the archive and streamed to the client, with the `plain/text` content type. The artifact file provides more detail than what is available in the [CSV export](../user/application_security/vulnerability_report/index.md#export-vulnerability-details). diff --git a/doc/api/users.md b/doc/api/users.md index 0671ebfe2b1..31fe6234ad2 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -911,7 +911,7 @@ Example response: ``` Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) also see these -preferences: +preferences if `code_suggestions_used_by_default` feature flag is disabled: - `code_suggestions` @@ -949,7 +949,7 @@ Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) also ca | Attribute | Required | Description | |:---------------------------------|:---------|:---------------------------------------------------| -| `code_suggestions` | No | Flag indicating the user allows code suggestions. Argument is experimental and can be removed in the future without notice. | +| `code_suggestions` | No | Flag indicating the user allows code suggestions. Argument is experimental and can be removed in the future without notice. In GitLab 16.8 and later, this attribute is ignored if `code_suggestions_used_by_default` feature flag is enabled. | ## User follow diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md index f20f2fd5e3f..e7c286a507b 100644 --- a/doc/ci/components/index.md +++ b/doc/ci/components/index.md @@ -68,8 +68,13 @@ The repository must contain: - In sub-directories containing `template.yml` files as entry points, for components that bundle together multiple related files. For example, `templates/secret-detection/template.yml`. -Configure the project's `.gitlab-ci.yml` to [test the components](#test-the-component) -and [release new versions](#publish-a-new-release). +You should also: + +- Configure the project's `.gitlab-ci.yml` to [test the components](#test-the-component) + and [release new versions](#publish-a-new-release). +- Add a `LICENSE.md` file with a license of your choice that covers the usage of your component. + For example the [MIT](https://opensource.org/license/mit/) or [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0#apply) + open source licenses. For example: @@ -78,6 +83,7 @@ For example: ```plaintext ├── templates/ │ └── my-component.yml + ├── LICENSE.md ├── README.md └── .gitlab-ci.yml ``` @@ -91,6 +97,7 @@ For example: │ ├── template.yml │ ├── Dockerfile │ └── test.sh + ├── LICENSE.md ├── README.md └── .gitlab-ci.yml ``` diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md index 7ad264f5823..6593335b63d 100644 --- a/doc/ci/jobs/job_artifacts.md +++ b/doc/ci/jobs/job_artifacts.md @@ -227,17 +227,21 @@ unless the report is added as a regular artifact with `artifacts:paths`. You can download the artifacts archive for a specific job with a publicly accessible URL for the [job artifacts API](../../api/job_artifacts.md#download-the-artifacts-archive). -For example, to download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: +For example: -```plaintext -https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build -``` +- To download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: -For example, to download the file `review/index.html` from the latest job named `build` in the `main` branch of a project on GitLab.com: + ```plaintext + https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build + ``` -```plaintext -https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/raw/review/index.html?job=build -``` +- To download the file `review/index.html` from the latest job named `build` in the `main` branch of a project on GitLab.com: + + ```plaintext + https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/raw/review/index.html?job=build + ``` + + Files returned by this endpoint always have the `plain/text` content type. In both examples, replace `<project-id>` with a valid project ID, found at the top of the project details page. diff --git a/doc/development/architecture.md b/doc/development/architecture.md index dd5d83212d4..2f6f93fc15b 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -12,6 +12,7 @@ There are two software distributions of GitLab: - The open source [Community Edition](https://gitlab.com/gitlab-org/gitlab-foss/) (CE). - The open core [Enterprise Edition](https://gitlab.com/gitlab-org/gitlab/) (EE). + **Note:** The EE repository has been archived. GitLab now operates [under a single codebase](https://about.gitlab.com/blog/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/). GitLab is available under [different subscriptions](https://about.gitlab.com/pricing/). diff --git a/doc/development/internal_analytics/service_ping/index.md b/doc/development/internal_analytics/service_ping/index.md index fdeb1de2b4c..eb0e384b10d 100644 --- a/doc/development/internal_analytics/service_ping/index.md +++ b/doc/development/internal_analytics/service_ping/index.md @@ -361,7 +361,7 @@ Rake tasks exist to export Service Ping data in different formats. - The Rake tasks export the Redis counter class or the line of the Redis block for `redis_usage_data`. - The Rake tasks calculate the `alt_usage_data` metrics. -In the home directory of your local GitLab installation run the following Rake tasks for the YAML and JSON versions respectively: +In the home directory of your local GitLab installation run the following Rake tasks for either the YAML or the JSON versions: ```shell # for YAML export of SQL queries diff --git a/doc/devsecops.md b/doc/devsecops.md index 118606409ed..de87cf42a95 100644 --- a/doc/devsecops.md +++ b/doc/devsecops.md @@ -42,19 +42,19 @@ For details, see [this article about DevSecOps](https://about.gitlab.com/topics/ If your organization is facing any of the following challenges, a DevSecOps approach might be for you. -- **Development, security, and operations teams are siloed.** +- **Development, security, and operations teams are siloed.** If development and operations are isolated from security issues, they can't build secure software. And if security teams aren't part of the development process, they can't identify risks proactively. DevSecOps brings teams together to improve workflows and share ideas. Organizations might even see improved employee morale and retention. -- **Long development cycles are making it difficult to meet customer or stakeholder demands.** +- **Long development cycles are making it difficult to meet customer or stakeholder demands.** One reason for the struggle could be security. DevSecOps implements security at every step of - the development lifecycle, meaning that solid security doesn’t require the whole process to come to a halt. + the development lifecycle, meaning that solid security doesn't require the whole process to come to a halt. -- **You’re migrating to the cloud (or considering it).** +- **You're migrating to the cloud (or considering it).** Moving to the cloud often means bringing on new development processes, tools, and systems. - It’s a great time to make processes faster and more secure — and DevSecOps could make that a lot easier. + It's a great time to make processes faster and more secure — and DevSecOps could make that a lot easier. To get started with DevSecOps, [learn more, and try GitLab Ultimate for free](https://about.gitlab.com/solutions/security-compliance/). diff --git a/doc/integration/jira/troubleshooting.md b/doc/integration/jira/troubleshooting.md index 0487c20af74..6c8b49b4159 100644 --- a/doc/integration/jira/troubleshooting.md +++ b/doc/integration/jira/troubleshooting.md @@ -4,15 +4,11 @@ group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Troubleshooting Jira **(FREE ALL)** +# Troubleshooting Jira issue integration **(FREE ALL)** -This page contains a list of common issues you might encounter when working with Jira integrations. +This page contains a list of common issues you might encounter when working with the [Jira issue integration](configure.md). -## Jira issue integration - -When working with the [Jira issue integration](configure.md), you might encounter the following issues. - -### GitLab cannot link to a Jira issue +## GitLab cannot link to a Jira issue When you mention a Jira issue ID in GitLab, the issue link might be missing. [`sidekiq.log`](../../administration/logs/index.md#sidekiq-logs) might contain the following exception: @@ -23,7 +19,7 @@ No Link Issue Permission for issue 'JIRA-1234' To resolve this issue, ensure the Jira user you created for the [Jira issue integration](configure.md) has permission to link issues. -### GitLab cannot comment on a Jira issue +## GitLab cannot comment on a Jira issue If GitLab cannot comment on a Jira issue, ensure the Jira user you created for the [Jira issue integration](configure.md) has permission to: @@ -38,7 +34,7 @@ For the root cause, check the [`integrations_json.log`](../../administration/log In GitLab 16.1 and later, when an error occurs, the `integrations_json.log` file contains `client_*` keys in the outgoing API request to Jira. You can use the `client_*` keys to check the [Atlassian API documentation](https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issues/#api-group-issues) for why the error has occurred. -In the following example, Jira responds with a `404`. This error might happen if: +In the following example, Jira responds with a `404 Not Found`. This error might happen if: - The Jira user you created for the Jira issue integration does not have permission to view the issue. - The Jira issue ID you specified does not exist. @@ -58,7 +54,7 @@ In the following example, Jira responds with a `404`. This error might happen if For more information about returned status codes, see the [Jira Cloud platform REST API documentation](https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issues/#api-rest-api-2-issue-issueidorkey-get-response). -#### Using `curl` to verify access to a Jira issue +### Using `curl` to verify access to a Jira issue To verify that a Jira user can access a specific Jira issue, run the following script: @@ -66,9 +62,9 @@ To verify that a Jira user can access a specific Jira issue, run the following s curl --verbose --user "$USER:$API_TOKEN" "https://$ATLASSIAN_SUBDOMAIN.atlassian.net/rest/api/2/issue/$JIRA_ISSUE" ``` -If the user can access the issue, Jira responds with a `200` and the returned JSON includes the Jira issue details. +If the user can access the issue, Jira responds with a `200 OK` and the returned JSON includes the Jira issue details. -### GitLab cannot close a Jira issue +## GitLab cannot close a Jira issue If GitLab cannot close a Jira issue: @@ -79,25 +75,25 @@ If GitLab cannot close a Jira issue: - Check the Jira issue resolution field is not set. - Check the issue is not struck through in Jira lists. -### CAPTCHA after failed sign-in attempts +## CAPTCHA after failed sign-in attempts -CAPTCHA might be triggered after several consecutive failed sign-in attempts, -which might lead to a `401 unauthorized` error when testing your Jira integration. -If CAPTCHA has been triggered, you can't use the Jira REST API to -authenticate with the Jira site. +CAPTCHA might be triggered after consecutive failed sign-in attempts. +These failed attempts might lead to a `401 Unauthorized` when testing the Jira issue integration settings. +If CAPTCHA has been triggered, you cannot use the Jira REST API +to authenticate with the Jira site. -To fix this error, sign in to your Jira instance -and complete the CAPTCHA. +To resolve this issue, sign in to your Jira instance and complete the CAPTCHA. -### Integration does not work for an imported project +## Integration does not work for an imported project -There is a [known bug](https://gitlab.com/gitlab-org/gitlab/-/issues/341571) -where the Jira integration sometimes does not work for a project that has been imported. -As a workaround, disable the integration and then re-enable it. +The Jira issue integration might not work for a project that has been imported. +For more information, see [issue 341571](https://gitlab.com/gitlab-org/gitlab/-/issues/341571). -### `certificate verify failed` when testing or using the Jira issue integration +To resolve this issue, disable and then re-enable the integration. -When testing the Jira integration settings, you might see the following error: +## `certificate verify failed` when testing the integration settings + +When testing the Jira issue integration settings, you might get the following error: ```plaintext Connection failed. Check your integration settings. SSL_connect returned=1 errno=0 peeraddr=<jira.example.com> state=error: certificate verify failed (unable to get local issuer certificate) @@ -115,19 +111,18 @@ This error might also appear in the [`integrations_json.log`](../../administrati } ``` -The error occurs because the Jira certificate isn't publicly trusted, or the certificate chain is incomplete. Until this is resolved, GitLab won't connect to Jira. - -There can be other variations of this error message that are listed on the [common SSL errors](https://docs.gitlab.com/omnibus/settings/ssl/ssl_troubleshooting.html#common-ssl-errors) page. +The error occurs because the Jira certificate is not publicly trusted or the certificate chain is incomplete. +Until this issue is resolved, GitLab does not connect to Jira. -To resolve this, refer to the -[common SSL errors](https://docs.gitlab.com/omnibus/settings/ssl/ssl_troubleshooting.html#common-ssl-errors) page. +To resolve this issue, see +[Common SSL errors](https://docs.gitlab.com/omnibus/settings/ssl/ssl_troubleshooting.html#common-ssl-errors). -### Change all Jira projects to instance-level or group-level values +## Change all Jira projects to instance-level or group-level values WARNING: Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. -#### Change all projects instance wide +### Change all projects on the instance To change all Jira projects to use instance-level integration settings: @@ -158,7 +153,7 @@ To change all Jira projects to use instance-level integration settings: 1. Modify and save the instance-level integration from the UI to propagate the changes to all group-level and project-level integrations. -#### Change all projects in a group +### Change all projects in a group To change all Jira projects in a group (and its subgroups) to use group-level integration settings: @@ -168,7 +163,7 @@ To change all Jira projects in a group (and its subgroups) to use group-level in def reset_integration(target) integration = target.integrations.find_by(type: Integrations::Jira) - return if integration.nil? # Skip if the project has no Jira integration + return if integration.nil? # Skip if the project has no Jira issue integration return unless integration.inherit_from_id.nil? # Skip integrations that are already inheriting default_integration = Integration.default_integration(integration.type, target) @@ -194,7 +189,7 @@ To change all Jira projects in a group (and its subgroups) to use group-level in end ``` -### Update the Jira integration password for all projects +## Update the Jira issue integration password for all projects WARNING: Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. @@ -227,7 +222,7 @@ If that's the case, ensure the [**Due date** field is visible for issues](https: ### `An error occurred while requesting data from Jira` -When you try to view the Jira issue list in GitLab, you might see this message: +When you try to view the Jira issue list in GitLab, you might get this message: ```plaintext An error occurred while requesting data from Jira @@ -244,7 +239,7 @@ Your Jira project key must not have [restricted words and characters](https://co ### Jira credentials not allowed to access the data -When you try to view the Jira issue list in GitLab, you might see this message: +When you try to view the Jira issue list in GitLab, you might get this message: ```plaintext The credentials for accessing Jira are not allowed to access the data. Check your Jira integration credentials and try again. diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md index bf687811e94..a55c7b54346 100644 --- a/doc/operations/feature_flags.md +++ b/doc/operations/feature_flags.md @@ -431,7 +431,7 @@ The polling rate is configurable in SDKs. Provided that all clients are requesti For applications looking for more scalable solution, you should use [Unleash Proxy](#unleash-proxy-example). On GitLab.com, you should use Unleash Proxy to reduce the chance of being rate limited across endpoints. This proxy server sits between the server and clients. It makes requests to the server on behalf of the client groups, -so the number of outbound requests can be greatly reduced. +so the number of outbound requests can be greatly reduced. If you still get `429` responses, increase the `UNLEASH_FETCH_INTERVAL` value in the Unleash Proxy. There is also an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/295472) to give more capacity to the current rate limit. diff --git a/doc/user/application_security/dast/authentication.md b/doc/user/application_security/dast/authentication.md index 9948a257d2e..854526c4235 100644 --- a/doc/user/application_security/dast/authentication.md +++ b/doc/user/application_security/dast/authentication.md @@ -404,191 +404,4 @@ dast: ## Troubleshooting -The [logs](#read-the-logs) provide insight into what DAST is doing and expecting during the authentication process. For more detailed -information, configure the [authentication report](#configure-the-authentication-report). - -For more information about particular error messages or situations see [known problems](#known-problems). - -The browser-based analyzer is used to authenticate the user. For advanced troubleshooting, see [browser-based troubleshooting](browser_based_troubleshooting.md). - -### Read the logs - -The console output of the DAST CI/CD job shows information about the authentication process using the `AUTH` log module. -For example, the following log shows failed authentication for a multi-step login form. -Authentication failed because a home page should be displayed after login. Instead, the login form was still present. - -```plaintext -2022-11-16T13:43:02.000 INF AUTH attempting to authenticate -2022-11-16T13:43:02.000 INF AUTH loading login page LoginURL=https://example.com/login -2022-11-16T13:43:10.000 INF AUTH multi-step authentication detected -2022-11-16T13:43:15.000 INF AUTH verifying if user submit was successful true_when="HTTP status code < 400" -2022-11-16T13:43:15.000 INF AUTH requirement is satisfied, no login HTTP message detected want="HTTP status code < 400" -2022-11-16T13:43:20.000 INF AUTH verifying if login attempt was successful true_when="HTTP status code < 400 and has authentication token and no login form found (no element found when searching using selector css:[id=email] or css:[id=password] or css:[id=submit])" -2022-11-24T14:43:20.000 INF AUTH requirement is satisfied, HTTP login request returned status code 200 url=https://example.com/user/login?error=invalid%20credentials want="HTTP status code < 400" -2022-11-16T13:43:21.000 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[id=email] or css:[id=password] or css:[id=submit])" -2022-11-16T13:43:21.000 INF AUTH login attempt failed error="authentication failed: failed to authenticate user" -``` - -### Configure the authentication report - -WARNING: -The authentication report can contain sensitive information such as the credentials used to perform the login. - -An authentication report can be saved as a CI/CD job artifact to assist with understanding the cause of an authentication failure. - -The report contains steps performed during the login process, HTTP requests and responses, the Document Object Model (DOM) and screenshots. - -![dast-auth-report](img/dast_auth_report.jpg) - -An example configuration where the authentication debug report is exported may look like the following: - -```yaml -dast: - variables: - DAST_WEBSITE: "https://example.com" - DAST_AUTH_REPORT: "true" - artifacts: - paths: [gl-dast-debug-auth-report.html] - when: always -``` - -### Known problems - -#### Login form not found - -DAST failed to find a login form when loading the login page, often because the authentication URL could not be loaded. -The log reports a fatal error such as: - -```plaintext -2022-12-07T12:44:02.838 INF AUTH loading login page LoginURL=[authentication URL] -2022-12-07T12:44:11.119 FTL MAIN authentication failed: login form not found -``` - -Suggested actions: - -- Generate the [authentication report](#configure-the-authentication-report) to inspect HTTP response. -- Check the target application authentication is deployed and running. -- Check the `DAST_AUTH_URL` is correct. -- Check the GitLab Runner can access the `DAST_AUTH_URL`. -- Check the `DAST_BROWSER_PATH_TO_LOGIN_FORM` is valid if used. - -#### Scan doesn't crawl authenticated pages - -If DAST captures the wrong [authentication tokens](#authentication-tokens) during the authentication process then -the scan can't crawl authenticated pages. Names of cookies and storage authentication tokens are written to the log. For example: - -```plaintext -2022-11-24T14:42:31.492 INF AUTH authentication token cookies names=["sessionID"] -2022-11-24T14:42:31.492 INF AUTH authentication token storage events keys=["token"] -``` - -Suggested actions: - -- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the login worked as expected. -- Verify the logged authentication tokens are those used by your application. -- If using cookies to store authentication tokens, set the names of the authentication token cookies using `DAST_AUTH_COOKIES`. - -#### Unable to find elements with selector - -DAST failed to find the username, password, first submit button, or submit button elements. The log reports a fatal error such as: - -```plaintext -2022-12-07T13:14:11.545 FTL MAIN authentication failed: unable to find elements with selector: css:#username -``` - -Suggested actions: - -- Generate the [authentication report](#configure-the-authentication-report) to use the screenshot from the `Login page` to verify that the page loaded correctly. -- Load the login page in a browser and verify the [selectors](#finding-an-elements-selector) configured in `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, `DAST_FIRST_SUBMIT_FIELD`, and `DAST_SUBMIT_FIELD` are correct. - -#### Failed to authenticate user - -DAST failed to authenticate due to a failed login verification check. The log reports a fatal error such as: - -```plaintext -2022-12-07T06:39:49.483 INF AUTH verifying if login attempt was successful true_when="HTTP status code < 400 and has authentication token and no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" -2022-12-07T06:39:49.484 INF AUTH requirement is satisfied, HTTP login request returned status code 303 url=http://auth-manual:8090/login want="HTTP status code < 400" -2022-12-07T06:39:49.513 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" -2022-12-07T06:39:49.589 INF AUTH login attempt failed error="authentication failed: failed to authenticate user" -2022-12-07T06:39:53.626 FTL MAIN authentication failed: failed to authenticate user -``` - -Suggested actions: - -- Look in the log for the `requirement is unsatisfied`. Respond to the appropriate error. - -#### Requirement unsatisfied, login form was found - -Applications typically display a dashboard when the user logs in and the login form with an error message when the -username or password is incorrect. - -This error occurs when DAST detects the login form on the page displayed after authenticating the user, -indicating that the login attempt failed. - -```plaintext -2022-12-07T06:39:49.513 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" -``` - -Suggested actions: - -- Verify that the username and password/authentication credentials used are correct. -- Generate the [authentication report](#configure-the-authentication-report) and verify the `Request` for the `Login submit` is correct. -- It's possible that the authentication report `Login submit` request and response are empty. This occurs when there is no request that would result - in a full page reload, such as a request made when submitting a HTML form. This occurs when using websockets or AJAX to submit the login form. -- If the page displayed following user authentication genuinely has elements matching the login form selectors, configure `DAST_AUTH_VERIFICATION_URL` - or `DAST_AUTH_VERIFICATION_SELECTOR` to use an alternate method of verifying the login attempt. - -#### Requirement unsatisfied, selector returned no results - -DAST cannot find an element matching the selector provided in `DAST_AUTH_VERIFICATION_SELECTOR` on the page displayed following user login. - -```plaintext -2022-12-07T06:39:33.239 INF AUTH requirement is unsatisfied, searching DOM using selector returned no results want="has element css:[name=welcome]" -``` - -Suggested actions: - -- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the expected page is displayed. -- Ensure the `DAST_AUTH_VERIFICATION_SELECTOR` [selector](#finding-an-elements-selector) is correct. - -#### Requirement unsatisfied, browser not at URL - -DAST detected that the page displayed following user login has a URL different to what was expected according to `DAST_AUTH_VERIFICATION_URL`. - -```plaintext -2022-12-07T11:28:00.241 INF AUTH requirement is unsatisfied, browser is not at URL browser_url="https://example.com/home" want="is at url https://example.com/user/dashboard" -``` - -Suggested actions: - -- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the expected page is displayed. -- Ensure the `DAST_AUTH_VERIFICATION_URL` is correct. - -#### Requirement unsatisfied, HTTP login request status code - -The HTTP response when loading the login form or submitting the form had a status code of 400 (client error) -or 500 (server error). - -```plaintext -2022-12-07T06:39:53.626 INF AUTH requirement is unsatisfied, HTTP login request returned status code 502 url="https://example.com/user/login" want="HTTP status code < 400" -``` - -- Verify that the username and password/authentication credentials used are correct. -- Generate the [authentication report](#configure-the-authentication-report) and verify the `Request` for the `Login submit` is correct. -- Verify the target application works as expected. - -#### Requirement unsatisfied, no authentication token - -DAST could not detect an [authentication token](#authentication-tokens) created during the authentication process. - -```plaintext -2022-12-07T11:25:29.010 INF AUTH authentication token cookies names=[] -2022-12-07T11:25:29.010 INF AUTH authentication token storage events keys=[] -2022-12-07T11:25:29.010 INF AUTH requirement is unsatisfied, no basic authentication, cookie or storage event authentication token detected want="has authentication token" -``` - -Suggestion actions: - -- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the login worked as expected. -- Using the browser's developer tools, investigate the cookies and local/session storage objects created while logging in. Ensure there is an authentication token created with sufficiently random value. -- If using cookies to store authentication tokens, set the names of the authentication token cookies using `DAST_AUTH_COOKIES`. +See [troubleshooting](authentication_troubleshooting.md) for more information. diff --git a/doc/user/application_security/dast/authentication_troubleshooting.md b/doc/user/application_security/dast/authentication_troubleshooting.md new file mode 100644 index 00000000000..62894d89ff6 --- /dev/null +++ b/doc/user/application_security/dast/authentication_troubleshooting.md @@ -0,0 +1,196 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Troubleshooting **(ULTIMATE ALL)** + +The [logs](#read-the-logs) provide insight into what DAST is doing and expecting during the authentication process. For more detailed +information, configure the [authentication report](#configure-the-authentication-report). + +For more information about particular error messages or situations see [known problems](#known-problems). + +The browser-based analyzer is used to authenticate the user. For advanced troubleshooting, see [browser-based troubleshooting](browser_based_troubleshooting.md). + +## Read the logs + +The console output of the DAST CI/CD job shows information about the authentication process using the `AUTH` log module. +For example, the following log shows failed authentication for a multi-step login form. +Authentication failed because a home page should be displayed after login. Instead, the login form was still present. + +```plaintext +2022-11-16T13:43:02.000 INF AUTH attempting to authenticate +2022-11-16T13:43:02.000 INF AUTH loading login page LoginURL=https://example.com/login +2022-11-16T13:43:10.000 INF AUTH multi-step authentication detected +2022-11-16T13:43:15.000 INF AUTH verifying if user submit was successful true_when="HTTP status code < 400" +2022-11-16T13:43:15.000 INF AUTH requirement is satisfied, no login HTTP message detected want="HTTP status code < 400" +2022-11-16T13:43:20.000 INF AUTH verifying if login attempt was successful true_when="HTTP status code < 400 and has authentication token and no login form found (no element found when searching using selector css:[id=email] or css:[id=password] or css:[id=submit])" +2022-11-24T14:43:20.000 INF AUTH requirement is satisfied, HTTP login request returned status code 200 url=https://example.com/user/login?error=invalid%20credentials want="HTTP status code < 400" +2022-11-16T13:43:21.000 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[id=email] or css:[id=password] or css:[id=submit])" +2022-11-16T13:43:21.000 INF AUTH login attempt failed error="authentication failed: failed to authenticate user" +``` + +## Configure the authentication report + +WARNING: +The authentication report can contain sensitive information such as the credentials used to perform the login. + +An authentication report can be saved as a CI/CD job artifact to assist with understanding the cause of an authentication failure. + +The report contains steps performed during the login process, HTTP requests and responses, the Document Object Model (DOM) and screenshots. + +![dast-auth-report](img/dast_auth_report.jpg) + +An example configuration where the authentication debug report is exported may look like the following: + +```yaml +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_AUTH_REPORT: "true" + artifacts: + paths: [gl-dast-debug-auth-report.html] + when: always +``` + +## Known problems + +### Login form not found + +DAST failed to find a login form when loading the login page, often because the authentication URL could not be loaded. +The log reports a fatal error such as: + +```plaintext +2022-12-07T12:44:02.838 INF AUTH loading login page LoginURL=[authentication URL] +2022-12-07T12:44:11.119 FTL MAIN authentication failed: login form not found +``` + +Suggested actions: + +- Generate the [authentication report](#configure-the-authentication-report) to inspect HTTP response. +- Check the target application authentication is deployed and running. +- Check the `DAST_AUTH_URL` is correct. +- Check the GitLab Runner can access the `DAST_AUTH_URL`. +- Check the `DAST_BROWSER_PATH_TO_LOGIN_FORM` is valid if used. + +### Scan doesn't crawl authenticated pages + +If DAST captures the wrong [authentication tokens](authentication.md#authentication-tokens) during the authentication process then +the scan can't crawl authenticated pages. Names of cookies and storage authentication tokens are written to the log. For example: + +```plaintext +2022-11-24T14:42:31.492 INF AUTH authentication token cookies names=["sessionID"] +2022-11-24T14:42:31.492 INF AUTH authentication token storage events keys=["token"] +``` + +Suggested actions: + +- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the login worked as expected. +- Verify the logged authentication tokens are those used by your application. +- If using cookies to store authentication tokens, set the names of the authentication token cookies using `DAST_AUTH_COOKIES`. + +### Unable to find elements with selector + +DAST failed to find the username, password, first submit button, or submit button elements. The log reports a fatal error such as: + +```plaintext +2022-12-07T13:14:11.545 FTL MAIN authentication failed: unable to find elements with selector: css:#username +``` + +Suggested actions: + +- Generate the [authentication report](#configure-the-authentication-report) to use the screenshot from the `Login page` to verify that the page loaded correctly. +- Load the login page in a browser and verify the [selectors](authentication.md#finding-an-elements-selector) configured in `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, `DAST_FIRST_SUBMIT_FIELD`, and `DAST_SUBMIT_FIELD` are correct. + +### Failed to authenticate user + +DAST failed to authenticate due to a failed login verification check. The log reports a fatal error such as: + +```plaintext +2022-12-07T06:39:49.483 INF AUTH verifying if login attempt was successful true_when="HTTP status code < 400 and has authentication token and no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" +2022-12-07T06:39:49.484 INF AUTH requirement is satisfied, HTTP login request returned status code 303 url=http://auth-manual:8090/login want="HTTP status code < 400" +2022-12-07T06:39:49.513 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" +2022-12-07T06:39:49.589 INF AUTH login attempt failed error="authentication failed: failed to authenticate user" +2022-12-07T06:39:53.626 FTL MAIN authentication failed: failed to authenticate user +``` + +Suggested actions: + +- Look in the log for the `requirement is unsatisfied`. Respond to the appropriate error. + +### Requirement unsatisfied, login form was found + +Applications typically display a dashboard when the user logs in and the login form with an error message when the +username or password is incorrect. + +This error occurs when DAST detects the login form on the page displayed after authenticating the user, +indicating that the login attempt failed. + +```plaintext +2022-12-07T06:39:49.513 INF AUTH requirement is unsatisfied, login form was found want="no login form found (no element found when searching using selector css:[name=username] or css:[name=password] or css:button[type=\"submit\"])" +``` + +Suggested actions: + +- Verify that the username and password/authentication credentials used are correct. +- Generate the [authentication report](#configure-the-authentication-report) and verify the `Request` for the `Login submit` is correct. +- It's possible that the authentication report `Login submit` request and response are empty. This occurs when there is no request that would result + in a full page reload, such as a request made when submitting a HTML form. This occurs when using websockets or AJAX to submit the login form. +- If the page displayed following user authentication genuinely has elements matching the login form selectors, configure `DAST_AUTH_VERIFICATION_URL` + or `DAST_AUTH_VERIFICATION_SELECTOR` to use an alternate method of verifying the login attempt. + +### Requirement unsatisfied, selector returned no results + +DAST cannot find an element matching the selector provided in `DAST_AUTH_VERIFICATION_SELECTOR` on the page displayed following user login. + +```plaintext +2022-12-07T06:39:33.239 INF AUTH requirement is unsatisfied, searching DOM using selector returned no results want="has element css:[name=welcome]" +``` + +Suggested actions: + +- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the expected page is displayed. +- Ensure the `DAST_AUTH_VERIFICATION_SELECTOR` [selector](authentication.md#finding-an-elements-selector) is correct. + +### Requirement unsatisfied, browser not at URL + +DAST detected that the page displayed following user login has a URL different to what was expected according to `DAST_AUTH_VERIFICATION_URL`. + +```plaintext +2022-12-07T11:28:00.241 INF AUTH requirement is unsatisfied, browser is not at URL browser_url="https://example.com/home" want="is at url https://example.com/user/dashboard" +``` + +Suggested actions: + +- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the expected page is displayed. +- Ensure the `DAST_AUTH_VERIFICATION_URL` is correct. + +### Requirement unsatisfied, HTTP login request status code + +The HTTP response when loading the login form or submitting the form had a status code of 400 (client error) +or 500 (server error). + +```plaintext +2022-12-07T06:39:53.626 INF AUTH requirement is unsatisfied, HTTP login request returned status code 502 url="https://example.com/user/login" want="HTTP status code < 400" +``` + +- Verify that the username and password/authentication credentials used are correct. +- Generate the [authentication report](#configure-the-authentication-report) and verify the `Request` for the `Login submit` is correct. +- Verify the target application works as expected. + +### Requirement unsatisfied, no authentication token + +DAST could not detect an [authentication token](authentication.md#authentication-tokens) created during the authentication process. + +```plaintext +2022-12-07T11:25:29.010 INF AUTH authentication token cookies names=[] +2022-12-07T11:25:29.010 INF AUTH authentication token storage events keys=[] +2022-12-07T11:25:29.010 INF AUTH requirement is unsatisfied, no basic authentication, cookie or storage event authentication token detected want="has authentication token" +``` + +Suggestion actions: + +- Generate the [authentication report](#configure-the-authentication-report) and look at the screenshot from the `Login submit` to verify that the login worked as expected. +- Using the browser's developer tools, investigate the cookies and local/session storage objects created while logging in. Ensure there is an authentication token created with sufficiently random value. +- If using cookies to store authentication tokens, set the names of the authentication token cookies using `DAST_AUTH_COOKIES`. diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 01473237a63..bd40209320d 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -116,7 +116,7 @@ Implementation options for SPPs differ slightly between GitLab SaaS and GitLab s main difference is that on GitLab SaaS it's only possible to create subgroups. Ensuring separation of duties requires more granular permission configuration. -### Enforce policies across subgroups and projects **(ULTIMATE SAAS) +### Enforce policies across subgroups and projects **(ULTIMATE SAAS)** To enforce policies against subgroups and projects, create a subgroup to contain the SPPs, separate to the subgroups containing the projects. Using separate subgroups allows for separation of duties, diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 7be4f3e927a..7a94db939cc 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -232,6 +232,18 @@ To install a package from the instance level, the package must have been publish npm install @scope/my-package ``` +### Package forwarding to npmjs.com + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/55344) in GitLab 12.9. + +When an npm package is not found in the package registry, GitLab responds with an HTTP redirect so the requesting client can resend the request to [npmjs.com](https://www.npmjs.com/). + +Administrators can disable this behavior in the [Continuous Integration settings](../../../administration/settings/continuous_integration.md). + +Group owners can disable this behavior in the group **Packages and registries** settings. + +Improvements are tracked in [epic 3608](https://gitlab.com/groups/gitlab-org/-/epics/3608). + ## Deprecate a package > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396763) in GitLab 16.0. @@ -267,14 +279,6 @@ npm deprecate @scope/package "" ## Helpful hints -### Package forwarding to npmjs.com - -When an npm package is not found in the package registry, the request is forwarded to [npmjs.com](https://www.npmjs.com/). The forward is performed by sending an HTTP redirect back to the requesting client. - -Administrators can disable this behavior in the [Continuous Integration settings](../../../administration/settings/continuous_integration.md). - -Group owners can disable this behavior in the group Packages and Registries settings. - ### Install npm packages from other organizations You can route package requests to organizations and users outside of GitLab. diff --git a/doc/user/project/repository/code_suggestions/self_managed.md b/doc/user/project/repository/code_suggestions/self_managed.md index 524a15193ae..26850bc8b5f 100644 --- a/doc/user/project/repository/code_suggestions/self_managed.md +++ b/doc/user/project/repository/code_suggestions/self_managed.md @@ -65,6 +65,9 @@ The users in your instance can now use Code Suggestions. ### GitLab 16.2 and earlier +FLAG: +On self-managed GitLab 16.0 and earlier, GitLab Duo Code Suggestions is not available. To use this feature, you must have GitLab 16.1 or later. For optimal performance and full feature access, you should upgrade to GitLab 16.3 or later, which supports cloud licensing. + Prerequisites: - You are an administrator. |