diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-11-14 11:41:52 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-11-14 11:41:52 +0300 |
| commit | 585826cb22ecea5998a2c2a4675735c94bdeedac (patch) | |
| tree | 5b05f0b30d33cef48963609e8a18a4dff260eab3 /doc/user | |
| parent | df221d036e5d0c6c0ee4d55b9c97f481ee05dee8 (diff) | |
Add latest changes from gitlab-org/gitlab@16-6-stable-eev16.6.0-rc42
Diffstat (limited to 'doc/user')
152 files changed, 2239 insertions, 999 deletions
diff --git a/doc/user/ai_features.md b/doc/user/ai_features.md index e24d50efee1..222752a4561 100644 --- a/doc/user/ai_features.md +++ b/doc/user/ai_features.md @@ -7,43 +7,37 @@ type: index, reference # GitLab Duo +> - [First GitLab Duo features introduced](https://about.gitlab.com/blog/2023/05/03/gitlab-ai-assisted-features/) in GitLab 16.0. +> - [Removed third-party AI setting](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/136144) in GitLab 16.6. +> - [Removed support for OpenAI from all GitLab Duo features](https://gitlab.com/groups/gitlab-org/-/epics/10964) in GitLab 16.6. + GitLab is creating AI-assisted features across our DevSecOps platform. These features aim to help increase velocity and solve key pain points across the software development lifecycle. | Feature | Purpose | Large Language Model | Current availability | Maturity | |-|-|-|-|-| | [Suggested Reviewers](project/merge_requests/reviews/index.md#gitlab-duo-suggested-reviewers) | Assists in creating faster and higher-quality reviews by automatically suggesting reviewers for your merge request. | GitLab creates a machine learning model for each project, which is used to generate reviewers <br><br> [View the issue](https://gitlab.com/gitlab-org/modelops/applied-ml/applied-ml-updates/-/issues/10) | SaaS only <br><br> Ultimate tier | [Generally Available (GA)](../policy/experiment-beta-support.md#generally-available-ga) | -| [Code Suggestions](project/repository/code_suggestions/index.md) | Helps you write code more efficiently by viewing code suggestions as you type. | [`code-gecko`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-completion) and [`code-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-generation) <br><br> [Anthropic's Claude](https://www.anthropic.com/product) model | SaaS <br> Self-managed <br><br> All tiers | [Beta](../policy/experiment-beta-support.md#beta) | -| [Vulnerability summary](application_security/vulnerabilities/index.md#explaining-a-vulnerability) | Helps you remediate vulnerabilities more efficiently, uplevel your skills, and write more secure code. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) <br><br> Anthropic's claude model if degraded performance | SaaS only <br><br> Ultimate tier | [Beta](../policy/experiment-beta-support.md#beta) | -| [Code explanation](#explain-code-in-the-web-ui-with-code-explanation) | Helps you understand code by explaining it in English language. | [`codechat-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-chat) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [GitLab Duo Chat](#answer-questions-with-gitlab-duo-chat) | Process and generate text and code in a conversational manner. Helps you quickly identify useful information in large volumes of text in issues, epics, code, and GitLab documentation. | Anthropic's claude model <br><br> OpenAI Embeddings | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Code Suggestions](project/repository/code_suggestions/index.md) | Helps you write code more efficiently by viewing code suggestions as you type. | For Code Completion: Vertex AI Codey [`code-gecko`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-completion) <br><br> For Code Generation: Anthropic [`Claude-2`](https://docs.anthropic.com/claude/reference/selecting-a-model)| [SaaS: All tiers](project/repository/code_suggestions/saas.md) <br><br> [Self-managed: Premium and Ultimate with Cloud Licensing](project/repository/code_suggestions/self_managed.md) | [Beta](../policy/experiment-beta-support.md#beta) | +| [Vulnerability summary](application_security/vulnerabilities/index.md#explaining-a-vulnerability) | Helps you remediate vulnerabilities more efficiently, boost your skills, and write more secure code. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) <br><br> Anthropic [`Claude-2`](https://docs.anthropic.com/claude/reference/selecting-a-model) if degraded performance | SaaS only <br><br> Ultimate tier | [Beta](../policy/experiment-beta-support.md#beta) | +| [Code explanation](#explain-code-in-the-web-ui-with-code-explanation) | Helps you understand code by explaining it in English language. | Vertex AI Codey [`codechat-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-chat) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [GitLab Duo Chat](gitlab_duo_chat.md) | Process and generate text and code in a conversational manner. Helps you quickly identify useful information in large volumes of text in issues, epics, code, and GitLab documentation. | Anthropic [`Claude-2`](https://docs.anthropic.com/claude/reference/selecting-a-model) <br><br> Vertex AI Codey [`textembedding-gecko`](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | | [Value stream forecasting](#forecast-deployment-frequency-with-value-stream-forecasting) | Assists you with predicting productivity metrics and identifying anomalies across your software development lifecycle. | Statistical forecasting | SaaS only <br> Self-managed <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Discussion summary](#summarize-issue-discussions-with-discussion-summary) | Assists with quickly getting everyone up to speed on lengthy conversations to help ensure you are all on the same page. | OpenAI's GPT-3 | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Merge request summary](project/merge_requests/ai_in_merge_requests.md#summarize-merge-request-changes) | Efficiently communicate the impact of your merge request changes. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Code review summary](project/merge_requests/ai_in_merge_requests.md#summarize-my-merge-request-review) | Helps ease merge request handoff between authors and reviewers and help reviewers efficiently understand suggestions. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Merge request template population](project/merge_requests/ai_in_merge_requests.md#fill-in-merge-request-templates) | Generate a description for the merge request based on the contents of the template. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Test generation](project/merge_requests/ai_in_merge_requests.md#generate-suggested-tests-in-merge-requests) | Automates repetitive tasks and helps catch bugs early. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Git suggestions](https://gitlab.com/gitlab-org/gitlab/-/issues/409636) | Helps you discover or recall Git commands when and where you need them. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Root cause analysis](#root-cause-analysis) | Assists you in determining the root cause for a pipeline failure and failed CI/CD build. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Issue description generation](#summarize-an-issue-with-issue-description-generation) | Generate issue descriptions. | OpenAI's GPT-3 | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Discussion summary](#summarize-issue-discussions-with-discussion-summary) | Assists with quickly getting everyone up to speed on lengthy conversations to help ensure you are all on the same page. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Merge request summary](project/merge_requests/ai_in_merge_requests.md#summarize-merge-request-changes) | Efficiently communicate the impact of your merge request changes. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Code review summary](project/merge_requests/ai_in_merge_requests.md#summarize-my-merge-request-review) | Helps ease merge request handoff between authors and reviewers and help reviewers efficiently understand suggestions. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Merge request template population](project/merge_requests/ai_in_merge_requests.md#fill-in-merge-request-templates) | Generate a description for the merge request based on the contents of the template. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Test generation](project/merge_requests/ai_in_merge_requests.md#generate-suggested-tests-in-merge-requests) | Automates repetitive tasks and helps catch bugs early. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Git suggestions](https://gitlab.com/gitlab-org/gitlab/-/issues/409636) | Helps you discover or recall Git commands when and where you need them. | Vertex AI Codey [`codechat-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-chat) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Root cause analysis](#root-cause-analysis) | Assists you in determining the root cause for a pipeline failure and failed CI/CD build. | Vertex AI Codey [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Issue description generation](#summarize-an-issue-with-issue-description-generation) | Generate issue descriptions. | Anthropic [`Claude-2`](https://docs.anthropic.com/claude/reference/selecting-a-model) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | ## Enable AI/ML features -- Third-party AI features - - All features built on large language models (LLM) from Google, - Anthropic or OpenAI (besides Code Suggestions) require that this setting is - enabled at the group level. - - [Generally Available](../policy/experiment-beta-support.md#generally-available-ga) - features are available when third-party AI features are enabled. - - Third-party AI features are enabled by default. - - This setting is available to Ultimate groups on SaaS and can be - set by a user who has the Owner role in the group. - - View [how to enable this setting](group/manage.md#enable-third-party-ai-features). - Experiment and Beta features - All features categorized as [Experiment features](../policy/experiment-beta-support.md#experiment) or [Beta features](../policy/experiment-beta-support.md#beta) (besides Code Suggestions) require that this setting is enabled at the group - level. This is in addition to the Third-party AI features setting. + level. - Their usage is subject to the [Testing Terms of Use](https://about.gitlab.com/handbook/legal/testing-agreement/). - Experiment and Beta features are disabled by default. @@ -65,7 +59,6 @@ The following subsections describe the experimental AI features in more detail. To use this feature: - The parent group of the project must: - - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). - You must be a member of the project with sufficient permissions to view the repository. @@ -104,52 +97,6 @@ code in a merge request: We cannot guarantee that the large language model produces results that are correct. Use the explanation with caution. -### Answer questions with GitLab Duo Chat **(ULTIMATE SAAS EXPERIMENT)** - -> Introduced in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). - -To use this feature, at least one group you're a member of must: - -- Have the [third-party AI features setting](group/manage.md#enable-third-party-ai-features) enabled. -- Have the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features) enabled. - -You can get AI generated support from GitLab Duo Chat about the following topics: - -- How to use GitLab. -- Questions about an issue. -- Summarizing an issue. - -Example questions you might ask: - -- `What is a fork?` -- `How to reset my password` -- `Summarize the issue <link to your issue>` -- `Summarize the description of the current issue` - -The examples above all use data from either the issue or the GitLab documentation. However, you can also ask to generate code, CI/CD configurations, or to explain code. For example: - -- `Write a hello world function in Ruby` -- `Write a tic tac toe game in JavaScript` -- `Write a .gitlab-ci.yml file to test and build a rails application` -- `Explain the following code: def sum(a, b) a + b end` - -You can also ask follow-up questions. - -This is an experimental feature and we're continuously extending the capabilities and reliability of the chat. - -1. In the lower-left corner, select the Help icon. - The [new left sidebar must be enabled](../tutorials/left_sidebar/index.md#enable-the-new-left-sidebar). -1. Select **Ask in GitLab Duo Chat**. A drawer opens on the right side of your screen. -1. Enter your question in the chat input box and press **Enter** or select **Send**. It may take a few seconds for the interactive AI chat to produce an answer. -1. You can ask a follow-up question. -1. If you want to ask a new question unrelated to the previous conversation, you may receive better answers if you clear the context by typing `/reset` into the input box and selecting **Send**. - -To give feedback about a specific response, use the feedback buttons in the response message. -Or, you can add a comment in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/415591). - -NOTE: -Only the last 50 messages are retained in the chat history. The chat history expires 3 days after last use. - ### Summarize issue discussions with Discussion summary **(ULTIMATE SAAS EXPERIMENT)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10344) in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). @@ -157,7 +104,6 @@ Only the last 50 messages are retained in the chat history. The chat history exp To use this feature: - The parent group of the issue must: - - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). - You must be a member of the project with sufficient permissions to view the issue. @@ -181,7 +127,6 @@ language model referenced above. To use this feature: - The parent group of the project must: - - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). - You must be a member of the project with sufficient permissions to view the CI/CD analytics. @@ -207,7 +152,6 @@ Provide feedback on this experimental feature in [issue 416833](https://gitlab.c To use this feature: - The parent group of the project must: - - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). - You must be a member of the project with sufficient permissions to view the CI/CD job. @@ -222,7 +166,6 @@ reason for the failure. To use this feature: - The parent group of the project must: - - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). - You must be a member of the project with sufficient permissions to view the issue. @@ -239,9 +182,13 @@ Provide feedback on this experimental feature in [issue 409844](https://gitlab.c **Data usage**: When you use this feature, the text you enter is sent to the large language model referenced above. +### GitLab Duo Chat **(ULTIMATE SAAS EXPERIMENT)** + +For details about this Experimental feature, see [GitLab Duo Chat](gitlab_duo_chat.md). + ## Data usage -GitLab AI features leverage generative AI to help increase velocity and aim to help make you more productive. Each feature operates independently of other features and is not required for other features to function. +GitLab AI features leverage generative AI to help increase velocity and aim to help make you more productive. Each feature operates independently of other features and is not required for other features to function. GitLab selects the best-in-class large-language models for specific tasks. We use [Google Vertex AI Models](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview#genai-models) and [Anthropic Claude](https://www.anthropic.com/product). ### Progressive enhancement @@ -251,13 +198,38 @@ These features are designed as a progressive enhancement to existing GitLab feat These features are in a variety of [feature support levels](../policy/experiment-beta-support.md#beta). Due to the nature of these features, there may be high demand for usage which may cause degraded performance or unexpected downtime of the feature. We have built these features to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable **beta and experimental** features for any or all customers at any time at our discretion. -## Third party services - ### Data privacy -Some AI features require the use of third-party AI services models and APIs from: Google AI and OpenAI. The processing of any personal data is in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). You may also visit the [Sub-Processors page](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors) to see the list of our Sub-Processors that we use to provide these features. +GitLab Duo AI features are powered by a generative AI models. The processing of any personal data is in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). You may also visit the [Sub-Processors page](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors) to see the list of our Sub-Processors that we use to provide these features. + +### Data retention + +The below reflects the current retention periods of GitLab AI model [Sub-Processors](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors): + +- Anthropic retains input and output data for 30 days. +- Google discards input and output data immediately after the output is provided. Google currently does not store data for abuse monitoring. + +All of these AI providers are under data protection agreements with GitLab that prohibit the use of Customer Content for their own purposes, except to perform their independent legal obligations. + +### Telemetry + +GitLab Duo collects aggregated or de-identified first-party usage data through our [Snowplow collector](https://about.gitlab.com/handbook/business-technology/data-team/platform/snowplow/). This usage data includes the following metrics: + +- Number of unique users +- Number of unique instances +- Prompt lengths +- Model used +- Status code responses +- API responses times + +### Training data + +GitLab does not train generative AI models based on private (non-public) data. The vendors we work with also do not train models based on private data. + +For more information on our AI [sub-processors](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors), see: -Group owners can control which top-level groups have access to third-party AI features by using the [group level third-party AI features setting](group/manage.md#enable-third-party-ai-features). +- Google Vertex AI Models APIs [data governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance) and [responsible AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai). +- Anthropic Claude's [constitution](https://www.anthropic.com/index/claudes-constitution). ### Model accuracy and quality diff --git a/doc/user/analytics/analytics_dashboards.md b/doc/user/analytics/analytics_dashboards.md index 448a46fdc26..8bed8018eb8 100644 --- a/doc/user/analytics/analytics_dashboards.md +++ b/doc/user/analytics/analytics_dashboards.md @@ -39,6 +39,12 @@ When [product analytics](../product_analytics/index.md) is enabled and onboarded - **Audience** displays metrics related to traffic, such as the number of users and sessions. - **Behavior** displays metrics related to user activity, such as the number of page views and events. +For more information about the development of product analytics, see the [group direction page](https://about.gitlab.com/direction/analytics/product-analytics/). To leave feedback about bugs or functionality: + +- Comment on issue [391970](https://gitlab.com/gitlab-org/gitlab/-/issues/391970). +- Create an issue with the `group::product analytics` label. +- [Schedule a call](https://calendly.com/jheimbuck/30-minute-call) with the team. + ### Value Stream Management - **Value Streams Dashboard** displays metrics related to [DevOps performance, security exposure, and workstream optimization](../analytics/value_streams_dashboard.md#devsecops-metrics-comparison-panel). diff --git a/doc/user/analytics/dora_metrics.md b/doc/user/analytics/dora_metrics.md index 391a1c7965f..e90bfd690ca 100644 --- a/doc/user/analytics/dora_metrics.md +++ b/doc/user/analytics/dora_metrics.md @@ -65,9 +65,14 @@ For software leaders, Lead time for changes reflects the efficiency of CI/CD pip Over time, the lead time for changes should decrease, while your team's performance should increase. Low lead time for changes means more efficient CI/CD pipelines. In GitLab, Lead time for changes is measure by the `Median time it takes for a merge request to get merged into production (from master)`. +By default, Lead time for changes measures only one-branch operations with multiple deployment jobs (for example, jobs moving from development to staging to production jobs on the main branch). +When a merge request gets merged in staging and then merge to production, GitLab processes them as two deployed merge requests, not one. + ### How lead time for changes is calculated -GitLab calculates Lead time for changes base on the number of seconds to successfully deliver a commit into production - **from** code committed **to** code successfully running in production, without adding the `coding_time` to the calculation. +GitLab calculates Lead time for changes based on the number of seconds to successfully deliver a commit into production - **from** code committed **to** code successfully running in production, without adding the `coding_time` to the calculation. + +By default, Lead time for changes supports measuring only one branch operation with multiple deployment jobs (for example, from development to staging to production on the default branch). When a merge request gets merged on staging, and then on production, GitLab interprets them as two deployed merge requests, not one. ### How to improve lead time for changes @@ -127,41 +132,37 @@ To improve this metric, you should consider: - Improving the efficacy of code review processes. - Adding more automated testing. -## DORA metrics in GitLab +## DORA custom calculation rules **(ULTIMATE ALL EXPERIMENT)** -The DORA metrics are displayed on the following charts: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96561) in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `dora_configuration`. Disabled by default. This feature is an [Experiment](../../policy/experiment-beta-support.md). -- [Value Streams Dashboard](value_streams_dashboard.md), which helps you identify trends, patterns, and opportunities for improvement. DORA metrics are displayed in the [metrics comparison panel](value_streams_dashboard.md#devsecops-metrics-comparison-panel) and the [DORA Performers score panel](value_streams_dashboard.md#dora-performers-score-panel). -- [CI/CD analytics charts](ci_cd_analytics.md), which show pipeline success rates and duration, and the history of DORA metrics over time. -- Insights reports for [groups](../group/insights/index.md) and [projects](../group/value_stream_analytics/index.md), where you can also use [DORA query parameters](../../user/project/insights/index.md#dora-query-parameters) to create custom charts. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `dora_configuration`. +On GitLab.com, this feature is not available. -The table below provides an overview of the DORA metrics' data aggregation in different charts. +This feature is an [Experiment](../../policy/experiment-beta-support.md). +To join the list of users testing this feature, [here is a suggested test flow](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96561#steps-to-check-on-localhost). +If you find a bug, [open an issue here](https://gitlab.com/groups/gitlab-org/-/epics/11490). +To share your use cases and feedback, comment in [epic 11490](https://gitlab.com/groups/gitlab-org/-/epics/11490). -| Metric name | Measured values | Data aggregation in the [Value Streams Dashboard](value_streams_dashboard.md) | Data aggregation in [CI/CD analytics charts](ci_cd_analytics.md) | Data aggregation in [Custom insights reporting](../../user/project/insights/index.md#dora-query-parameters) | -|---------------------------|-------------------|-----------------------------------------------------|------------------------|----------| -| Deployment frequency | Number of successful deployments | daily average per month | daily average | `day` (default) or `month` | -| Lead time for changes | Number of seconds to successfully deliver a commit into production | daily median per month | median time | `day` (default) or `month` | -| Time to restore service | Number of seconds an incident was open for | daily median per month | daily median | `day` (default) or `month` | -| Change failure rate | percentage of deployments that cause an incident in production | daily median per month | percentage of failed deployments | `day` (default) or `month` | +### DORA Lead Time For Changes - multi-branch rule -## Configure DORA metrics calculation **(ULTIMATE ALL BETA)** +Unlike the default [calculation of Lead time for changes](#how-lead-time-for-changes-is-calculated), this calculation rule allows measuring multi-branch operations with a single deployment job for each operation. +For example, from development job on development branch, to staging job on staging branch, to production job on production branch. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96561) in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `dora_configuration`. Disabled by default. This feature is in [Beta](../../policy/experiment-beta-support.md). +This calculation rule has been implemented by updating the `dora_configurations` table with the target branches that are part of the development flow. +This way, GitLab can recognize the branches as one, and filter out other merge requests. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `dora_configuration`. -On GitLab.com, this feature is not available. -This feature is not ready for production use. +This configuration changes how daily DORA metrics are calculated for the selected project, but doesn't affect other projects, groups, or users. + +This feature supports only project-level propagation. -You can configure the behavior of DORA metrics calculations. To do this, in the Rails console run the following command: ```ruby Dora::Configuration.create!(project: my_project, ltfc_target_branches: \['master', 'main'\]) ``` -This feature is in [Beta](../../policy/experiment-beta-support.md). - ## Retrieve DORA metrics data To retrieve DORA data, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. @@ -193,7 +194,9 @@ and use it to automatically: 1. [Create an incident when an alert is triggered](../../operations/incident_management/manage_incidents.md#automatically-when-an-alert-is-triggered). 1. [Close incidents via recovery alerts](../../operations/incident_management/manage_incidents.md#automatically-close-incidents-via-recovery-alerts). -### Supported DORA metrics in GitLab +## DORA metrics in GitLab + +GitLab supports the following DORA metrics: | Metric | Level | API | UI chart | Comments | |---------------------------|-------------------|-----------------------------------------------------|------------------------|----------| @@ -203,3 +206,22 @@ and use it to automatically: | `lead_time_for_changes` | Group | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 14.0 and later | Unit in seconds. Aggregation method is median. | | `time_to_restore_service` | Project and group | [GitLab 14.9 and later](../../api/dora/metrics.md) | GitLab 15.1 and later | Unit in days. Aggregation method is median. | | `change_failure_rate` | Project and group | [GitLab 14.10 and later](../../api/dora/metrics.md) | GitLab 15.2 and later | Percentage of deployments. | + +### DORA metrics charts + +The DORA metrics are displayed on the following charts: + +- [Value Streams Dashboard](value_streams_dashboard.md), which helps you identify trends, patterns, and opportunities for improvement. DORA metrics are displayed in the [metrics comparison panel](value_streams_dashboard.md#devsecops-metrics-comparison-panel) and the [DORA Performers score panel](value_streams_dashboard.md#dora-performers-score-panel). +- [CI/CD analytics charts](ci_cd_analytics.md), which show pipeline success rates and duration, and the history of DORA metrics over time. +- Insights reports for [groups](../group/insights/index.md) and [projects](../group/value_stream_analytics/index.md), where you can also use [DORA query parameters](../../user/project/insights/index.md#dora-query-parameters) to create custom charts. + +### DORA metrics data aggregation + +The table below provides an overview of the DORA metrics' data aggregation in different charts. + +| Metric name | Measured values | Data aggregation in the [Value Streams Dashboard](value_streams_dashboard.md) | Data aggregation in [CI/CD analytics charts](ci_cd_analytics.md) | Data aggregation in [Custom insights reporting](../../user/project/insights/index.md#dora-query-parameters) | +|---------------------------|-------------------|-----------------------------------------------------|------------------------|----------| +| Deployment frequency | Number of successful deployments | daily average per month | daily average | `day` (default) or `month` | +| Lead time for changes | Number of seconds to successfully deliver a commit into production | daily median per month | median time | `day` (default) or `month` | +| Time to restore service | Number of seconds an incident was open for | daily median per month | daily median | `day` (default) or `month` | +| Change failure rate | percentage of deployments that cause an incident in production | daily median per month | percentage of failed deployments | `day` (default) or `month` | diff --git a/doc/user/analytics/value_streams_dashboard.md b/doc/user/analytics/value_streams_dashboard.md index 45be6f5aa25..b5358cc81c8 100644 --- a/doc/user/analytics/value_streams_dashboard.md +++ b/doc/user/analytics/value_streams_dashboard.md @@ -214,8 +214,8 @@ Label filters are appended as query parameters to the URL of the drill-down repo | Change failure rate | Percentage of deployments that cause an incident in production. | [Change failure rate tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=change-failure-rate) | [Change failure rate](dora_metrics.md#change-failure-rate) | `change_failure_rate` | | Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#lifecycle-metrics) | `lead_time` | | Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#lifecycle-metrics) | `cycle_time` | -| New issues | Number of new issues created. | [Issue Analytics](https://gitlab.com/groups/gitlab-org/-/issues_analytics) | Issue analytics [for projects](issue_analytics.md) and [for groups](../../user/group/issues_analytics/index.md) | `issues` | -| Closed issues | Number of issues closed by month. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [Value Stream Analytics](../group/value_stream_analytics/index.md) | `issues_completed` | +| Issues created | Number of new issues created. | [Issue Analytics](https://gitlab.com/groups/gitlab-org/-/issues_analytics) | Issue analytics [for projects](issue_analytics.md) and [for groups](../../user/group/issues_analytics/index.md) | `issues` | +| Issues closed | Number of issues closed by month. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [Value Stream Analytics](../group/value_stream_analytics/index.md) | `issues_completed` | | Number of deploys | Total number of deploys to production. | [Merge Request Analytics](https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics) | [Merge request analytics](merge_request_analytics.md) | `deploys` | | Merge request throughput | The number of merge requests merged by month. | [Groups Productivity analytics](productivity_analytics.md), [Projects Merge Request Analytics](https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics) | [Groups Productivity analytics](productivity_analytics.md) [Projects Merge request analytics](merge_request_analytics.md) | `merge_request_throughput` | | Critical vulnerabilities over time | Critical vulnerabilities over time in project or group | [Vulnerability report](https://gitlab.com/gitlab-org/gitlab/-/security/vulnerability_report) | [Vulnerability report](../application_security/vulnerability_report/index.md) | `vulnerability_critical` | diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 6ee8be822da..ac03f08e23b 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -7,11 +7,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Container Scanning **(FREE ALL)** -> - Improved support for FIPS [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263482) in GitLab 13.6 by upgrading `CS_MAJOR_VERSION` from `2` to `3`. -> - Integration with Trivy [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322656) in GitLab 13.9 by upgrading `CS_MAJOR_VERSION` from `3` to `4`. -> - Integration with Clair [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/321451) in GitLab 13.9. -> - Default container scanning with Trivy [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61850) in GitLab 14.0. -> - Integration with Grype as an alternative scanner [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326279) in GitLab 14.0. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86092) the major analyzer version from `4` to `5` in GitLab 15.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86783) from GitLab Ultimate to GitLab Free in 15.0. > - Container Scanning variables that reference Docker [renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/357264) in GitLab 15.4. @@ -22,8 +17,9 @@ vulnerabilities. By including an extra Container Scanning job in your pipeline t vulnerabilities and displays them in a merge request, you can use GitLab to audit your Docker-based apps. -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Container Scanning](https://www.youtube.com/watch?v=C0jn2eN5MAs). +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a video walkthrough, see [How to set up Container Scanning using GitLab](https://youtu.be/h__mcXpil_4?si=w_BVG68qnkL9x4l1). Container Scanning is often considered part of Software Composition Analysis (SCA). SCA can contain aspects of inspecting the items your code uses. These items typically include application and system @@ -58,23 +54,23 @@ information directly in the merge request. ### Capabilities -| Capability | In Free | In Ultimate | +| Capability | In Free and Premium | In Ultimate | | --- | ------ | ------ | -| [Configure Scanners](#configuration) | Yes | Yes | -| Customize Settings ([Variables](#available-cicd-variables), [Overriding](#overriding-the-container-scanning-template), [offline environment support](#running-container-scanning-in-an-offline-environment), etc) | Yes | Yes | -| [View JSON Report](#reports-json-format) as a CI job artifact | Yes | Yes | -| Generation of a JSON report of [dependencies](#dependency-list) as a CI job artifact | Yes | Yes | -| Ability to enable container scanning via an MR in the GitLab UI | Yes | Yes | -| [UBI Image Support](#fips-enabled-images) | Yes | Yes | -| Support for Trivy | Yes | Yes | -| Support for Grype | Yes | Yes | +| [Configure Scanners](#configuration) | **{check-circle}** Yes | **{check-circle}** Yes | +| Customize Settings ([Variables](#available-cicd-variables), [Overriding](#overriding-the-container-scanning-template), [offline environment support](#running-container-scanning-in-an-offline-environment), etc) | **{check-circle}** Yes | **{check-circle}** Yes | +| [View JSON Report](#reports-json-format) as a CI job artifact | **{check-circle}** Yes | **{check-circle}** Yes | +| Generation of a JSON report of [dependencies](#dependency-list) as a CI job artifact | **{check-circle}** Yes | **{check-circle}** Yes | +| Ability to enable container scanning via an MR in the GitLab UI | **{check-circle}** Yes | **{check-circle}** Yes | +| [UBI Image Support](#fips-enabled-images) | **{check-circle}** Yes | **{check-circle}** Yes | +| Support for Trivy | **{check-circle}** Yes | **{check-circle}** Yes | +| Support for Grype | **{check-circle}** Yes | **{check-circle}** Yes | | Inclusion of GitLab Advisory Database | Limited to the time-delayed content from GitLab [advisories-communities](https://gitlab.com/gitlab-org/advisories-community/) project | Yes - all the latest content from [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) | -| Presentation of Report data in Merge Request and Security tab of the CI pipeline job | No | Yes | -| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) such as merge request approvals | No | Yes | -| [Solutions for vulnerabilities (auto-remediation)](#solutions-for-vulnerabilities-auto-remediation) | No | Yes | -| Support for the [vulnerability allow list](#vulnerability-allowlisting) | No | Yes | -| [Access to Security Dashboard page](#security-dashboard) | No | Yes | -| [Access to Dependency List page](../dependency_list/index.md) | No | Yes | +| Presentation of Report data in Merge Request and Security tab of the CI pipeline job | **{dotted-circle}** No | **{check-circle}** Yes | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) such as merge request approvals | **{dotted-circle}** No | **{check-circle}** Yes | +| [Solutions for vulnerabilities (auto-remediation)](#solutions-for-vulnerabilities-auto-remediation) | **{dotted-circle}** No | **{check-circle}** Yes | +| Support for the [vulnerability allow list](#vulnerability-allowlisting) | **{dotted-circle}** No | **{check-circle}** Yes | +| [Access to Security Dashboard page](#security-dashboard) | **{dotted-circle}** No | **{check-circle}** Yes | +| [Access to Dependency List page](../dependency_list/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | ## Prerequisites @@ -133,6 +129,10 @@ Setting `CS_DEFAULT_BRANCH_IMAGE` avoids duplicate vulnerability findings when a The value of `CS_DEFAULT_BRANCH_IMAGE` indicates the name of the scanned image as it appears on the default branch. For more details on how this deduplication is achieved, see [Setting the default branch image](#setting-the-default-branch-image). +## Running jobs in merge request pipelines + +See [Use security scanning tools with merge request pipelines](../index.md#use-security-scanning-tools-with-merge-request-pipelines) + ### Customizing the container scanning settings There may be cases where you want to customize how GitLab scans your containers. For example, you @@ -272,28 +272,30 @@ including a large number of false positives. | `CS_REGISTRY_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. The default is only set if `$CS_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | | `CS_DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | All | | `CS_QUIET` | `""` | If set, this variable disables output of the [vulnerabilities table](#container-scanning-job-log-format) in the job log. [Introduced](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/merge_requests/50) in GitLab 15.1. | All | -| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | All | +| `CS_TRIVY_JAVA_DB` | `"ghcr.io/aquasecurity/trivy-java-db"` | Specify an alternate location for the [trivy-java-db](https://github.com/aquasecurity/trivy-java-db) vulnerability database. | Trivy | +| `CS_IGNORE_STATUSES` | `""` | Force the analyzer to ignore vulnerability findings with specified statuses in a comma-delimited list. For `trivy`, the following values are allowed: `unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life`. For `grype`, the following values are allowed: `fixed,not-fixed,unknown,wont-fix` | All | +| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. | All | ### Supported distributions Support depends on which scanner is used: -| Distribution | Grype | Trivy | -| -------------- | ----- | ----- | -| Alma Linux | | ✅ | -| Alpine Linux | ✅ | ✅ | -| Amazon Linux | ✅ | ✅ | -| BusyBox | ✅ | | -| CentOS | ✅ | ✅ | -| CBL-Mariner | | ✅ | -| Debian | ✅ | ✅ | -| Distroless | ✅ | ✅ | -| Oracle Linux | ✅ | ✅ | -| Photon OS | | ✅ | -| Red Hat (RHEL) | ✅ | ✅ | -| Rocky Linux | | ✅ | -| SUSE | | ✅ | -| Ubuntu | ✅ | ✅ | +| Distribution | Grype | Trivy | +|----------------|------------------------|------------------------| +| Alma Linux | **{dotted-circle}** No | **{check-circle}** Yes | +| Alpine Linux | **{check-circle}** Yes | **{check-circle}** Yes | +| Amazon Linux | **{check-circle}** Yes | **{check-circle}** Yes | +| BusyBox | **{check-circle}** Yes | **{dotted-circle}** No | +| CentOS | **{check-circle}** Yes | **{check-circle}** Yes | +| CBL-Mariner | **{dotted-circle}** No | **{check-circle}** Yes | +| Debian | **{check-circle}** Yes | **{check-circle}** Yes | +| Distroless | **{check-circle}** Yes | **{check-circle}** Yes | +| Oracle Linux | **{check-circle}** Yes | **{check-circle}** Yes | +| Photon OS | **{dotted-circle}** No | **{check-circle}** Yes | +| Red Hat (RHEL) | **{check-circle}** Yes | **{check-circle}** Yes | +| Rocky Linux | **{dotted-circle}** No | **{check-circle}** Yes | +| SUSE | **{dotted-circle}** No | **{check-circle}** Yes | +| Ubuntu | **{check-circle}** Yes | **{check-circle}** Yes | #### FIPS-enabled images @@ -654,6 +656,32 @@ Also: Scanning images in external private registries is not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. +#### Create and use a Trivy Java database mirror + +When the `trivy` scanner is used and a `jar` file is encountered in a container image being scanned, `trivy` downloads an additional `trivy-java-db` vulnerability database. By default, the `trivy-java-db` database is hosted as an [OCI artifact](https://oras.land/docs/quickstart) at `ghcr.io/aquasecurity/trivy-java-db:1`. If this registry is not accessible, for example in a network-isolated offline GitLab instance, one solution is to mirror the `trivy-java-db` to a container registry that can be accessed in the offline instance: + +```yaml +mirror trivy java db: + image: + name: ghcr.io/oras-project/oras:v1.1.0 + entrypoint: [""] + script: + - oras login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + - oras pull ghcr.io/aquasecurity/trivy-java-db:1 + - oras push $CI_REGISTRY_IMAGE:1 --config /dev/null:application/vnd.aquasec.trivy.config.v1+json javadb.tar.gz:application/vnd.aquasec.trivy.javadb.layer.v1.tar+gzip +``` + +If the above container registry is `gitlab.example.com/trivy-java-db-mirror`, then the container scanning job should be configured in the following way: + +```yaml +include: + - template: Security/Container-Scanning.gitlab-ci.yml + +container_scanning: + variables: + CS_TRIVY_JAVA_DB: gitlab.example.com/trivy-java-db-mirror:1 +``` + ## Running the standalone container scanning tool It's possible to run the [GitLab container scanning tool](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning) @@ -715,24 +743,24 @@ All analyzer images are [updated daily](https://gitlab.com/gitlab-org/security-p The images use data from upstream advisory databases depending on which scanner is used: -| Data Source | Trivy | Grype | -| ------------------------------ | ----- | ----- | -| AlmaLinux Security Advisory | ✅ | ✅ | -| Amazon Linux Security Center | ✅ | ✅ | -| Arch Linux Security Tracker | ✅ | | -| SUSE CVRF | ✅ | ✅ | -| CWE Advisories | ✅ | | -| Debian Security Bug Tracker | ✅ | ✅ | -| GitHub Security Advisory | ✅ | ✅ | -| Go Vulnerability Database | ✅ | | -| CBL-Mariner Vulnerability Data | ✅ | | -| NVD | ✅ | ✅ | -| OSV | ✅ | | -| Red Hat OVAL v2 | ✅ | ✅ | -| Red Hat Security Data API | ✅ | ✅ | -| Photon Security Advisories | ✅ | | -| Rocky Linux UpdateInfo | ✅ | | -| Ubuntu CVE Tracker (only data sources from mid 2021 and later) | ✅ | ✅ | +| Data Source | Trivy | Grype | +|----------------------------------------------------------------|------------------------|------------------------| +| AlmaLinux Security Advisory | **{check-circle}** Yes | **{check-circle}** Yes | +| Amazon Linux Security Center | **{check-circle}** Yes | **{check-circle}** Yes | +| Arch Linux Security Tracker | **{check-circle}** Yes | **{dotted-circle}** No | +| SUSE CVRF | **{check-circle}** Yes | **{check-circle}** Yes | +| CWE Advisories | **{check-circle}** Yes | **{dotted-circle}** No | +| Debian Security Bug Tracker | **{check-circle}** Yes | **{check-circle}** Yes | +| GitHub Security Advisory | **{check-circle}** Yes | **{check-circle}** Yes | +| Go Vulnerability Database | **{check-circle}** Yes | **{dotted-circle}** No | +| CBL-Mariner Vulnerability Data | **{check-circle}** Yes | **{dotted-circle}** No | +| NVD | **{check-circle}** Yes | **{check-circle}** Yes | +| OSV | **{check-circle}** Yes | **{dotted-circle}** No | +| Red Hat OVAL v2 | **{check-circle}** Yes | **{check-circle}** Yes | +| Red Hat Security Data API | **{check-circle}** Yes | **{check-circle}** Yes | +| Photon Security Advisories | **{check-circle}** Yes | **{dotted-circle}** No | +| Rocky Linux UpdateInfo | **{check-circle}** Yes | **{dotted-circle}** No | +| Ubuntu CVE Tracker (only data sources from mid 2021 and later) | **{check-circle}** Yes | **{check-circle}** Yes | In addition to the sources provided by these scanners, GitLab maintains the following vulnerability databases: diff --git a/doc/user/application_security/continuous_vulnerability_scanning/index.md b/doc/user/application_security/continuous_vulnerability_scanning/index.md index 4094a0add28..e31fc5f7eb0 100644 --- a/doc/user/application_security/continuous_vulnerability_scanning/index.md +++ b/doc/user/application_security/continuous_vulnerability_scanning/index.md @@ -29,10 +29,9 @@ To enable Continuous Vulnerability Scanning: - Enable the Continuous Vulnerability Scanning setting in the project's [security configuration](../configuration/index.md). - Enable [Dependency Scanning](../dependency_scanning/index.md#configuration) and ensure that its prerequisites are met. +- On GitLab self-managed only, you can [choose package registry metadata to synchronize](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. For this data synchronization to work, you must allow outbound network traffic from your GitLab instance to the domain `storage.googleapis.com`. If you have limited or no network connectivity then please refer to the documentation section [running in an offline environment](#running-in-an-offline-environment) for further guidance. -On GitLab self-managed only, you can [choose package registry metadata to sync](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. - -### Requirements for offline environments +### Running in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required to successfully scan CycloneDX reports for vulnerabilities. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index 26782c319b1..207db52ed71 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -66,6 +66,8 @@ See [checks](checks/index.md) for more information about individual checks. Active scans check for vulnerabilities by injecting attack payloads into HTTP requests recorded during the crawl phase of the scan. Active scans are disabled by default due to the nature of their probing attacks. +#### How active scans work + DAST analyzes each recorded HTTP request for injection locations, such as query values, header values, cookie values, form posts, and JSON string values. Attack payloads are injected into the injection location, forming a new request. DAST sends the request to the target application and uses the HTTP response to determine attack success. @@ -84,6 +86,12 @@ A simplified timing attack works as follows: 1. The target application is vulnerable if it executes the query parameter value as a system command without validation, for example, `system(params[:search])` 1. DAST creates a finding if the response time takes longer than 10 seconds. +#### Known issues + +Active scans do not use a browser to send HTTP requests in an effort to minimize scan time. + +Anti-CSRF tokens are not regenerated for attacks that submit forms. Please disable anti-CSRF tokens when running an active scan. + ## Getting started To run a DAST scan: @@ -167,7 +175,7 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | CI/CD variable | Type | Example | Description | |:--------------------------------------------|:---------------------------------------------------------|----------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `DAST_ADVERTISE_SCAN` | boolean | `true` | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. | +| `DAST_ADVERTISE_SCAN` | boolean | `true` | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. The header value starts with `GitLab DAST`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. | | `DAST_BROWSER_ACTION_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `800ms` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis after completing an action. | | `DAST_BROWSER_ACTION_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to complete an action. | | `DAST_BROWSER_ALLOWED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. Headers set using `DAST_REQUEST_HEADERS` are added to every request made to these hostnames. | @@ -209,7 +217,7 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_REQUEST_HEADERS` | string | `Cache-control:no-cache` | Set to a comma-separated list of request header names and values. | | `DAST_SKIP_TARGET_CHECK` | boolean | `true` | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. | | `DAST_TARGET_AVAILABILITY_TIMEOUT` | number | `60` | Time limit in seconds to wait for target availability. | -| `DAST_WEBSITE` | URL | `https://example.com` | The URL of the website to scan. | +| `DAST_WEBSITE` | URL | `https://example.com` | The URL of the target application to scan. | | `SECURE_ANALYZERS_PREFIX` | URL | `registry.organization.com` | Set the Docker registry base address from which to download the analyzer. | ## Managing scope @@ -281,22 +289,17 @@ dast: DAST_EXCLUDE_URLS: "https://my.site.com/user/logout" # don't visit this URL ``` -## Vulnerability detection +## Vulnerability check migration + +A migration is underway that changes the browser-based analyzer from using the proxy-based analyzer Zed Attack Proxy (ZAP) active vulnerability checks, to using GitLab-built active vulnerability checks. + +The browser-based analyzer continues to use a combination of proxy-based analyzer and GitLab-built vulnerability checks until the migration is complete. See [browser-based vulnerability checks](checks/index.md) for details of which checks have been migrated. -Vulnerability detection is gradually being migrated from the default Zed Attack Proxy (ZAP) solution -to the browser-based analyzer. For details of the vulnerability detection already migrated, see -[browser-based vulnerability checks](checks/index.md). +### Why browser-based scans produce different results to proxy-based scans -The crawler runs the target website in a browser with DAST/ZAP configured as the proxy server. This -ensures that all requests and responses made by the browser are passively scanned by DAST/ZAP. When -running a full scan, active vulnerability checks executed by DAST/ZAP do not use a browser. This -difference in how vulnerabilities are checked can cause issues that require certain features of the -target website to be disabled to ensure the scan works as intended. +Browser-based and proxy-based scans do not produce the same results because they use a different set of vulnerability checks. -For example, for a target website that contains forms with Anti-CSRF tokens, a passive scan works as -intended because the browser displays pages and forms as if a user is viewing the page. However, -active vulnerability checks that run in a full scan cannot submit forms containing Anti-CSRF tokens. -In such cases, we recommend you disable Anti-CSRF tokens when running a full scan. +The browser-based analyzer does not have an equivalent for proxy-based checks that create too many false positives, are not worth running because modern browsers don't allow the vulnerability to be exploited, or are no longer considered relevant. The browser-based analyzer includes checks that proxy-based analyzer does not. ## Managing scan time diff --git a/doc/user/application_security/dast/checks/89.1.md b/doc/user/application_security/dast/checks/89.1.md new file mode 100644 index 00000000000..ca7ff5e4593 --- /dev/null +++ b/doc/user/application_security/dast/checks/89.1.md @@ -0,0 +1,37 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# SQL Injection + +## Description + +It is possible to execute arbitrary SQL commands on the target application server's +backend database. +SQL Injection is a critical vulnerability that can lead to a data or system +compromise. + +## Remediation + +Always use parameterized queries when issuing requests to backend database systems. In +situations where dynamic queries must be created, never use direct user input, but +instead use a map or dictionary of valid values and resolve them using a user-supplied key. + +For example, some database drivers do not allow parameterized queries for `>` or `<` comparison +operators. In these cases, do not use a user supplied `>` or `<` value, but rather have the user +supply a `gt` or `lt` value. The alphabetical values are then used to look up the `>` and `<` +values to be used in the construction of the dynamic query. The same goes for other queries where +column or table names are required but can not be parameterized. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 89.1 | false | 89 | Active | high | + +## Links + +- [OWASP](https://owasp.org/www-community/attacks/SQL_Injection) +- [CWE](https://cwe.mitre.org/data/definitions/89.html) diff --git a/doc/user/application_security/dast/checks/917.1.md b/doc/user/application_security/dast/checks/917.1.md new file mode 100644 index 00000000000..68b9665e393 --- /dev/null +++ b/doc/user/application_security/dast/checks/917.1.md @@ -0,0 +1,33 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Expression Language Injection + +## Description + +It is possible to execute arbitrary Expression Language (EL) statements on the target +application server. EL injection is a critical severity vulnerability that can lead to +full system compromise. EL injection can occur when attacker-controlled data is used to construct +EL statements without neutralizing special characters. These special characters could modify the +intended EL statement prior to it being executed by an interpreter. + +## Remediation + +User-controlled data should always have special elements neutralized when used as part of +constructing Expression Language statements. Please consult the documentation for the EL +interpreter in use on how properly neutralize user controlled data. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 917.1 | false | 917 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/917.html) +- [OWASP](https://owasp.org/www-community/vulnerabilities/Expression_Language_Injection) +- [Expression Language Injection [PDF]](https://mindedsecurity.com/wp-content/uploads/2020/10/ExpressionLanguageInjection.pdf) diff --git a/doc/user/application_security/dast/checks/94.1.md b/doc/user/application_security/dast/checks/94.1.md new file mode 100644 index 00000000000..ec30b41c5e8 --- /dev/null +++ b/doc/user/application_security/dast/checks/94.1.md @@ -0,0 +1,53 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Server-side code injection (PHP) + +## Description + +The target application was found vulnerable to code injection. A malicious actor could inject arbitrary +PHP code to be executed on the server. This could lead to a full system compromise by accessing +stored secrets, injecting code to take over accounts, or executing OS commands. + +## Remediation + +Never pass user input directly into functions which evaluate string data as code, such as `eval`. +There is almost no benefit of passing string values to `eval`, as such the best recommendation is +to replace the current logic with more safe implementations of dynamically evaluating logic with +user input. One alternative is to use an `array()`, storing expected user inputs in an array +key, and use that key as a look up to execute functions: + +```php +$func_to_run = function() +{ + print('hello world'); +}; + +$function_map = array(); +$function_map["fn"] = $func_to_run; // store additional input to function mappings here + +$input = "fn"; + +// lookup "fn" as the key +if (array_key_exists($input, $function_map)) { + // run the $func_to_run that was stored in the "fn" array hash value. + $func = $function_map[$input]; + $func(); +} else { + print('invalid input'); +} +``` + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 94.1 | false | 94 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/94.html) +- [OWASP](https://owasp.org/www-community/attacks/Code_Injection) diff --git a/doc/user/application_security/dast/checks/94.2.md b/doc/user/application_security/dast/checks/94.2.md new file mode 100644 index 00000000000..666052807b5 --- /dev/null +++ b/doc/user/application_security/dast/checks/94.2.md @@ -0,0 +1,51 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Server-side code injection (Ruby) + +## Description + +The target application was found vulnerable to code injection. A malicious actor could inject arbitrary +Ruby code to be executed on the server. This could lead to a full system compromise by accessing +stored secrets, injecting code to take over accounts, or executing OS commands. + +## Remediation + +Never pass user input directly into functions which evaluate string data as code, such as `eval`, +`send`, `public_send`, `instance_eval` or `class_eval`. There is almost no benefit of passing string +values to these methods, as such the best recommendation is to replace the current logic with more safe +implementations of dynamically evaluating logic with user input. If using `send` or `public_send` ensure +the first argument is to a known, hardcoded method/symbol and does not come from user input. + +For `eval`, `instance_eval` and `class_eval`, user input should never be sent directly to these methods. +One alternative is to store functions or methods in a Hash that can be looked up using a key. If the key +exists, the function can be executed. + +```ruby +def func_to_run + puts 'hello world' +end + +input = 'fn' + +function_map = { fn: method(:func_to_run) } + +if function_map.key?(input.to_sym) + function_map[input.to_sym].call +else + puts 'invalid input' +end +``` + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 94.2 | false | 94 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/94.html) diff --git a/doc/user/application_security/dast/checks/94.3.md b/doc/user/application_security/dast/checks/94.3.md new file mode 100644 index 00000000000..772cdb1d3ea --- /dev/null +++ b/doc/user/application_security/dast/checks/94.3.md @@ -0,0 +1,45 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Server-side code injection (Python) + +## Description + +The target application was found vulnerable to code injection. A malicious actor could inject arbitrary +Python code to be executed on the server. This could lead to a full system compromise by accessing +stored secrets, injecting code to take over accounts, or executing OS commands. + +## Remediation + +Never pass user input directly into functions which evaluate string data as code, such as `eval`, +or `exec`. There is almost no benefit of passing string values to these methods, as such the best +recommendation is to replace the current logic with more safe implementations of dynamically evaluating +logic with user input. One alternative is to store functions or methods in a hashmap that can be looked +up using a key. If the key exists, the function can be executed. + +```python +def func_to_run(): + print('hello world') + +function_map = {'fn': func_to_run} + +input = 'fn' + +if input in function_map: + function_map[input]() +else: + print('invalid input') +``` + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 94.3 | false | 94 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/94.html) diff --git a/doc/user/application_security/dast/checks/943.1.md b/doc/user/application_security/dast/checks/943.1.md new file mode 100644 index 00000000000..debae65669a --- /dev/null +++ b/doc/user/application_security/dast/checks/943.1.md @@ -0,0 +1,30 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Improper neutralization of special elements in data query logic + +## Description + +The application generates a query intended to interact with MongoDB, +but it does not neutralize or incorrectly neutralizes special elements +that can modify the intended logic of the query. + +## Remediation + +Refactor find or search queries to use standard +filtering operators such as `$gt` or `$in` instead of broad operators such +as `$where`. If possible, disable the MongoDB JavaScript interface entirely. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 943.1 | false | 943 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/943.html) +- [Disabling MongoDB Server Side JS](https://www.mongodb.com/docs/manual/core/server-side-javascript/#std-label-disable-server-side-js) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index 4d41f08672e..c239fdb5e74 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -170,4 +170,10 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne | [113.1](113.1.md) | Improper Neutralization of CRLF Sequences in HTTP Headers | High | Active | | [22.1](22.1.md) | Improper limitation of a pathname to a restricted directory (Path traversal) | High | Active | | [611.1](611.1.md) | External XML Entity Injection (XXE) | High | Active | +| [89.1](89.1.md) | SQL Injection | High | Active | +| [917.1](917.1.md) | Expression Language Injection | High | Active | +| [94.1](94.1.md) | Server-side code injection (PHP) | High | Active | +| [94.2](94.2.md) | Server-side code injection (Ruby) | High | Active | +| [94.3](94.3.md) | Server-side code injection (Python) | High | Active | | [94.4](94.4.md) | Server-side code injection (NodeJS) | High | Active | +| [943.1](943.1.md) | Improper neutralization of special elements in data query logic | High | Active | diff --git a/doc/user/application_security/dast/proxy-based.md b/doc/user/application_security/dast/proxy-based.md index 230d8ef5ca3..9e59ecc64d9 100644 --- a/doc/user/application_security/dast/proxy-based.md +++ b/doc/user/application_security/dast/proxy-based.md @@ -11,11 +11,14 @@ The DAST proxy-based analyzer can be added to your [GitLab CI/CD](../../../ci/in This helps you discover vulnerabilities in web applications that do not use JavaScript heavily. For applications that do, see the [DAST browser-based analyzer](browser_based.md). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video walkthrough, see [How to set up Dynamic Application Security Testing (DAST) with GitLab](https://youtu.be/EiFE1QrUQfk?si=6rpgwgUpalw3ByiV). + WARNING: Do not run DAST scans against a production server. Not only can it perform *any* function that a user can, such as clicking buttons or submitting forms, but it may also trigger bugs, leading to modification or loss of production data. Only run DAST scans against a test server. -The analyzer uses the [OWASP Zed Attack Proxy](https://www.zaproxy.org/) (ZAP) to scan in two different ways: +The analyzer uses the [Software Security Project Zed Attack Proxy](https://www.zaproxy.org/) (ZAP) to scan in two different ways: - Passive scan only (default). DAST executes [ZAP's Baseline Scan](https://www.zaproxy.org/docs/docker/baseline-scan/) and doesn't @@ -382,7 +385,7 @@ including a large number of false positives. | `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | | `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. | | `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. | -| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. | +| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `false`. | | `DAST_TARGET_AVAILABILITY_TIMEOUT` <sup>1</sup> | number | Time limit in seconds to wait for target availability. | | `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. | | `DAST_XML_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the XML report written at the end of a scan. | diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index c04134de2b2..683ba6ad19b 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -6,11 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Dependency Scanning **(ULTIMATE ALL)** -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an interactive reading and how-to demo of this Dependency Scanning doc, see [How to use dependency scanning tutorial hands-on GitLab Application Security part 3](https://youtu.be/ii05cMbJ4xQ?feature=shared) -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) - Dependency Scanning analyzes your application's dependencies for known vulnerabilities. All dependencies are scanned, including transitive dependencies, also known as nested dependencies. @@ -33,12 +28,16 @@ we encourage you to use all of our security scanners. For a comparison of these  -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o). - WARNING: Dependency Scanning does not support runtime installation of compilers and interpreters. +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o) +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo of this Dependency Scanning documentation, see [How to use dependency scanning tutorial hands-on GitLab Application Security part 3](https://youtu.be/ii05cMbJ4xQ?feature=shared) +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For other interactive reading and how-to demos, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) + ## Supported languages and package managers The following languages and dependency managers are supported: @@ -230,7 +229,8 @@ table.supported-languages ul { <li> <a id="notes-regarding-supported-languages-and-package-managers-2"></a> <p> - Java 21 LTS is only available when using <a href="https://maven.apache.org/">Maven</a> or <a href="https://gradle.org/">Gradle</a>. Java 21 LTS for <a href="https://www.scala-sbt.org/">sbt</a> is not yet available and tracked in <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/421174">issue 421174</a>. It is not supported when <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. + Java 21 LTS for <a href="https://www.scala-sbt.org/">sbt</a> is limited to version 1.9.7. Support for more <a href="https://www.scala-sbt.org/">sbt</a> versions can be tracked in <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/430335">issue 430335</a>. + It is not supported when <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. </p> </li> <li> @@ -599,6 +599,10 @@ To enable dependency scanning: Pipelines now include a dependency scanning job. +### Running jobs in merge request pipelines + +See [Use security scanning tools with merge request pipelines](../index.md#use-security-scanning-tools-with-merge-request-pipelines) + ### Customizing analyzer behavior You can use CI/CD variables to customize dependency scanning behavior. @@ -1093,6 +1097,17 @@ variables: GRADLE_CLI_OPTS: "-Dhttps.proxyHost=squid-proxy -Dhttps.proxyPort=3128 -Dhttp.proxyHost=squid-proxy -Dhttp.proxyPort=3128 -Dhttp.nonProxyHosts=localhost" ``` +## Using a proxy with Maven projects + +Maven does not read the `HTTP(S)_PROXY` environment variables. + +To make the Maven dependency scanner use a proxy, you can specify the options using the `MAVEN_CLI_OPTS` CI/CD variable: + +```yaml +variables: + MAVEN_CLI_OPTS: "-DproxySet=true -Dhttps.proxyHost=squid-proxy -Dhttps.proxyPort=3128 -Dhttp.proxyHost=squid-proxy -Dhttp.proxyPort=3218" +``` + ## Specific settings for languages and package managers See the following sections for configuring specific languages and package managers. diff --git a/doc/user/application_security/get-started-security.md b/doc/user/application_security/get-started-security.md index 3e73fbc5955..6143dd59373 100644 --- a/doc/user/application_security/get-started-security.md +++ b/doc/user/application_security/get-started-security.md @@ -11,32 +11,42 @@ For an overview, see [Adopting GitLab application security](https://www.youtube. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) -The following steps help you get the most from GitLab application security tools. These steps are a recommended order of operations. You can choose to implement capabilities in a different order or omit features that do not apply to your specific needs. - -1. Enable [Secret Detection](secret_detection/index.md) and [Dependency Scanning](dependency_scanning/index.md) - to identify any leaked secrets and vulnerable packages in your codebase. - - - For all security scanners, enable them by updating your [`.gitlab-ci.yml`](../../ci/yaml/gitlab_ci_yaml.md) directly on your `default` branch. This creates a baseline scan of your `default` branch, which is necessary for - feature branch scans to be compared against. This allows [merge requests](../project/merge_requests/index.md) - to display only newly-introduced vulnerabilities. Otherwise, merge requests display every - vulnerability in the branch, regardless of whether it was introduced by a change in the branch. - - If you are after simplicity, enable only Secret Detection first. It only has one analyzer, - no build requirements, and relatively simple findings: is this a secret or not? - - It is good practice to enable Dependency Scanning early so you can start identifying existing - vulnerable packages in your codebase. -1. Let your team get comfortable with [vulnerability reports](vulnerability_report/index.md) and - establish a vulnerability triage workflow. -1. Consider creating [labels](../project/labels.md) and [issue boards](../project/issue_board.md) to +The following steps help introduce you to GitLab application security tools incrementally. +You can choose to enable features in a different order, or skip features that don't apply to your specific needs. +You should start with: + +- [Secret Detection](secret_detection/index.md), which works with all programming languages and creates understandable results. +- [Dependency Scanning](dependency_scanning/index.md), which finds known vulnerabilities in the dependencies your code uses. + +If it's your first time setting up GitLab security scanning, you should start with a single project. +After you've gotten familiar with how scanning works, you can then choose to: + +- Follow [the same steps](#recommended-steps) to enable scanning in more projects. +- [Enforce scanning](index.md#enforce-scan-execution) across more of your projects at once. + +## Recommended steps + +1. Choose a project to enable and test security features. Consider choosing a project: + - That uses your organization's typical programming languages and technologies, because some scanning features work differently across languages. + - Where you can try out new settings, like required approvals, without interrupting your team's daily work. + You could create a copy of a higher-traffic project for testing, or select a project that's not as busy. +1. Create a merge request to [enable Secret Detection](secret_detection/index.md#enable-secret-detection) and [enable Dependency Scanning](dependency_scanning/index.md#configuration) + to identify any leaked secrets and vulnerable packages in that project. + - Security scanners run in your project's [CI/CD pipelines](../../ci/pipelines/index.md). Creating a merge request to update your [`.gitlab-ci.yml`](../../ci/index.md#the-gitlab-ciyml-file) helps you check how the scanners work with your project before they start running in every pipeline. In the merge request, you can change relevant [Secret Detection settings](secret_detection/index.md#configure-scan-settings) or [Dependency Scanning settings](dependency_scanning/index.md#available-cicd-variables) to accommodate your project's layout or configuration. For example, you might choose to exclude a directory of third-party code from scanning. + - After you merge this MR to your [default branch](../project/repository/branches/default.md), the system creates a baseline scan. This scan identifies which vulnerabilities already exist on the default branch so [merge requests](../project/merge_requests/index.md) can highlight only newly-introduced problems. Without a baseline scan, merge requests display every + vulnerability in the branch, even if the vulnerability already exists on the default branch. +1. Let your team get comfortable with [viewing security findings in merge requests](index.md#view-security-scan-information) and the [vulnerability report](vulnerability_report/index.md). +1. Establish a vulnerability triage workflow. + - Consider creating [labels](../project/labels.md) and [issue boards](../project/issue_board.md) to help manage issues created from vulnerabilities. Issue boards allow all stakeholders to have a common view of all issues and track remediation progress. +1. Monitor the [Security Dashboard](security_dashboard/index.md) trends to gauge success in remediating existing vulnerabilities and preventing the introduction of new ones. 1. Enforce scheduled security scanning jobs by using a [scan execution policy](policies/scan-execution-policies.md). - These scheduled jobs run independently from any other security scans you may have defined in a compliance framework pipeline or in the project's `.gitlab-ci.yml` file. - Running regular dependency and [container scans](container_scanning/index.md) surface newly-discovered vulnerabilities that already exist in your repository. - Scheduled scans are most useful for projects or important branches with low development activity where pipeline scans are infrequent. 1. Create a [scan result policy](policies/index.md) to limit new vulnerabilities from being merged - into your `default` branch. -1. Monitor the [Security Dashboard](security_dashboard/index.md) trends to gauge success in - remediating existing vulnerabilities and preventing the introduction of new ones. + into your [default branch](../project/repository/branches/default.md). 1. Enable other scan types such as [SAST](sast/index.md), [DAST](dast/index.md), [Fuzz testing](coverage_fuzzing/index.md), or [Container Scanning](container_scanning/index.md). 1. Use [Compliance Pipelines](../group/compliance_frameworks.md#compliance-pipelines) diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 62155e07fbc..25fa1f5cbaf 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -177,6 +177,9 @@ By default, the application security jobs are configured to run for branch pipel To use them with [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md), you must reference the [`latest` templates](../../development/cicd/templates.md). +The latest version of the template may include breaking changes. Use the stable template unless you +need a feature provided only in the latest template. + All `latest` security templates support merge request pipelines. For example, to run both SAST and Dependency Scanning, the following template is used: @@ -193,6 +196,9 @@ Mixing `latest` and `stable` security templates can cause both MR and branch pip NOTE: Latest templates can receive breaking changes in any release. +For more information about template versioning, see the +[CI/CD documentation](../../development/cicd/templates.md#latest-version). + ## Default behavior of GitLab security scanning tools ### Secure jobs in your pipeline @@ -264,7 +270,7 @@ In the Free tier, the reports above aren't parsed by GitLab. As a result, the wi A merge request contains a security widget which displays a summary of the _new_ results. New results are determined by comparing the findings of the merge request against the findings of the most recent completed pipeline (`success`, `failed`, `canceled` or `skipped`) for the commit when the feature branch was created from the target branch. -If security scans have not run for the completed pipeline in the target branch when the feature branch was created, there is no base for comparison. The vulnerabilities from the merge request findings are listed as new in the merge request security widget. We recommend you run a scan of the `default` (target) branch before enabling feature branch scans for your developers. +GitLab checks the last 10 pipelines for the commit when the feature was created from the target branch to find one with security reports to use in comparison logic. If security scans have not run for the last 10 completed pipelines in the target branch when the feature branch was created, there is no base for comparison. The vulnerabilities from the merge request findings are listed as new in the merge request security widget. We recommend you run a scan of the `default` (target) branch before enabling feature branch scans for your developers. The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both new and existing findings. @@ -472,9 +478,9 @@ You can always find supported and deprecated schema versions in the [source code You can interact with the results of the security scanning tools in several locations: - [Scan information in merge requests](#merge-request) -- [Project Security Dashboard](security_dashboard/index.md#view-vulnerabilities-over-time-for-a-project) +- [Project Security Dashboard](security_dashboard/index.md#project-security-dashboard) - [Security pipeline tab](security_dashboard/index.md) -- [Group Security Dashboard](security_dashboard/index.md#view-vulnerabilities-over-time-for-a-group) +- [Group Security Dashboard](security_dashboard/index.md#group-security-dashboard) - [Security Center](security_dashboard/index.md#security-center) - [Vulnerability Report](vulnerability_report/index.md) - [Vulnerability Pages](vulnerabilities/index.md) diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 0eb2355beb7..f6ef8a2c49e 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -28,9 +28,6 @@ implicitly so that the policies can be enforced. This ensures policies enabling secret detection, static analysis, or other scanners that do not require a build in the project, are still able to execute and be enforced. -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Enforcing scan execution policies on projects with no GitLab CI/CD configuration](https://www.youtube.com/watch?v=sUfwQQ4-qHs). - In the event of a job name collision, GitLab appends a hyphen and a number to the job name. GitLab increments the number until the name no longer conflicts with existing job names. If you create a policy at the group level, it applies to every child project or subgroup. You cannot edit a @@ -46,6 +43,9 @@ Policy jobs for scans other than DAST scans are created in the `test` stage of t [`stages`](../../../ci/yaml/index.md#stages), to remove the `test` stage, jobs will run in the `scan-policies` stage instead. This stage is injected into the CI pipeline at evaluation time if it doesn't exist. If the `build` stage exists, it is injected just after the `build` stage. If the `build` stage does not exist, it is injected at the beginning of the pipeline. DAST scans always run in the `dast` stage. If this stage does not exist, then a `dast` stage is injected at the end of the pipeline. +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a video walkthrough, see [How to set up Security Scan Policies in GitLab](https://youtu.be/ZBcqGmEwORA?si=aeT4EXtmHjosgjBY). +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Enforcing scan execution policies on projects with no GitLab CI/CD configuration](https://www.youtube.com/watch?v=sUfwQQ4-qHs). + ## Scan execution policy editor NOTE: diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index d892012c365..d0d3cb2ca03 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -27,13 +27,18 @@ The following video gives you an overview of GitLab scan result policies: <iframe src="https://www.youtube-nocookie.com/embed/w5I9gcUgr9U" frameborder="0" allowfullscreen> </iframe> </figure> +## Requirements and limitations + +- You must add the respective [security scanning tools](../index.md#application-coverage). + Otherwise, scan result policies do not have any effect. +- The maximum number of policies is five. +- Each policy can have a maximum of five rules. +- All configured scanners must be present in the merge request's latest pipeline. If not, approvals are required even if some vulnerability criteria have not been met. + ## Merge request with multiple pipelines > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/379108) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `multi_pipeline_scan_result_policies`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/409482) in GitLab 16.3. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `multi_pipeline_scan_result_policies`. On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/409482) in GitLab 16.3. Feature flag `multi_pipeline_scan_result_policies` removed. A project can have multiple pipeline types configured. A single commit can initiate multiple pipelines, each of which may contain a security scan. @@ -78,36 +83,31 @@ When you save a new policy, GitLab validates its contents against [this JSON sch If you're not familiar with how to read [JSON schemas](https://json-schema.org/), the following sections and tables provide an alternative. -| Field | Type | Required | Possible values | Description | -|-------|------|----------|-----------------|-------------| -| `scan_result_policy` | `array` of Scan Result Policy | true | | List of scan result policies (maximum 5). | +| Field | Type | Required | Possible values | Description | +|----------------------|-------------------------------|----------|-----------------|------------------------------------------| +| `scan_result_policy` | `array` of Scan Result Policy | true | | List of scan result policies (maximum 5). | ## Scan result policy schema -> The `approval_settings` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418752) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `scan_result_policy_settings`. Disabled by default. +> The `approval_settings` fields was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418752) in GitLab 16.4 [with flags](../../../administration/feature_flags.md) named `scan_result_policies_block_unprotecting_branches`, `scan_result_any_merge_request`, or `scan_result_policies_block_force_push`. All are disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `scan_result_policy_settings`. -On GitLab.com, this feature is not available. +On self-managed GitLab, by default the `approval_settings` field is unavailable. To show the feature, an administrator can [enable the feature flags](../../../administration/feature_flags.md) named `scan_result_policies_block_unprotecting_branches`, `scan_result_any_merge_request`, or `scan_result_policies_block_force_push`. See the `approval_settings` section below for more information. | Field | Type | Required |Possible values | Description | -|-------|------|----------|----------------|-------------| +|--------|------|----------|----------------|-------------| | `name` | `string` | true | | Name of the policy. Maximum of 255 characters.| -| `description` (optional) | `string` | true | | Description of the policy. | +| `description` | `string` | false | | Description of the policy. | | `enabled` | `boolean` | true | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | | `rules` | `array` of rules | true | | List of rules that the policy applies. | -| `actions` | `array` of actions | false| | List of actions that the policy enforces. | -| `approval_settings` | `object` | false | `{prevent_approval_by_author: boolean, prevent_approval_by_commit_author: boolean, remove_approvals_with_new_commit: boolean, require_password_to_approve: boolean}` | Project settings that the policy overrides. | +| `actions` | `array` of actions | false | | List of actions that the policy enforces. | +| `approval_settings` | `object` | false | | Project settings that the policy overrides. | ## `scan_finding` rule type -> - The scan result policy field `vulnerability_attributes` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123052) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `enforce_vulnerability_attributes_rules`. [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/418784) in GitLab 16.3. Feature flag `enforce_vulnerability_attributes_rules` removed in GitLab 16.5. +> - The scan result policy field `vulnerability_attributes` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123052) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `enforce_vulnerability_attributes_rules`. [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/418784) in GitLab 16.3. Feature flag removed. > - The scan result policy field `vulnerability_age` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123956) in GitLab 16.2. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Generally available in GitLab 16.5. Feature flag removed. - -FLAG: -On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. -On GitLab.com, this feature is available. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133753) in GitLab 16.5. Feature flag removed. This rule enforces the defined actions based on security scan findings. @@ -128,11 +128,7 @@ This rule enforces the defined actions based on security scan findings. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `license_scanning_policies`. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/397644) in GitLab 15.11. Feature flag `license_scanning_policies` removed. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. - -FLAG: -On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. -On GitLab.com, this feature is available. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133753) in GitLab 16.5. Feature flag removed. This rule enforces the defined actions based on license findings. @@ -148,12 +144,11 @@ This rule enforces the defined actions based on license findings. ## `any_merge_request` rule type -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418752) in GitLab 16.4. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133753) in GitLab 16.5. Feature flag removed. +> - The `any_merge_request` rule type was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418752) in GitLab 16.4. Disabled by default. FLAG: -On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. -On GitLab.com, this feature is available. +On self-managed GitLab, by default the `any_merge_request` field is not available. To show the feature, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `any_merge_request`. This rule enforces the defined actions for any merge request based on the commits signature. @@ -180,13 +175,28 @@ the defined policy. | `group_approvers_ids` | `array` of `integer` | false | ID of one of more groups | The IDs of groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | | `role_approvers` | `array` of `string` | false | One or more [roles](../../../user/permissions.md#roles) (for example: `owner`, `maintainer`) | The roles to consider as approvers that are eligible to approve. | -Requirements and limitations: +## `approval_settings` -- You must add the respective [security scanning tools](../index.md#application-coverage). - Otherwise, scan result policies do not have any effect. -- The maximum number of policies is five. -- Each policy can have a maximum of five rules. -- All configured scanners must be present in the merge request's latest pipeline. If not, approvals are required even if some vulnerability criteria have not been met. +> - The `block_unprotecting_branches` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/423101) in GitLab 16.4 [with flag](../../../administration/feature_flags.md) named `scan_result_policy_settings`. Disabled by default. +> - The `scan_result_policy_settings` feature flag was replaced by the `scan_result_policies_block_unprotecting_branches` feature flag in 16.4. +> - The `prevent_approval_by_author`, `prevent_approval_by_commit_author`, `remove_approvals_with_new_commit`, and `require_password_to_approve` fields were [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418752) in GitLab 16.4 [with flag](../../../administration/feature_flags.md) named `scan_result_any_merge_request`. Disabled by default. +> - The `prevent_force_pushing` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420629) in GitLab 16.4 [with flag](../../../administration/feature_flags.md) named `scan_result_policies_block_force_push`. Disabled by default. + +FLAG: +On self-managed GitLab, by default the `block_unprotecting_branches` field is unavailable. To show the feature, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `scan_result_policies_block_unprotecting_branches`. On GitLab.com, this feature is unavailable. +On self-managed GitLab, by default the `prevent_approval_by_author`, `prevent_approval_by_commit_author`, `remove_approvals_with_new_commit`, and `require_password_to_approve` fields are unavailable. To show the feature, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `scan_result_any_merge_request`. On GitLab.com, this feature is available. +On self-managed GitLab, by default the `prevent_force_pushing` field is unavailable. To show the feature, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. On GitLab.com, this feature is unavailable. + +The settings set in the policy overwrite settings in the project. + +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `block_unprotecting_branches` | `boolean` | false | `true`, `false` | Prevent a user from removing a branch from the protected branches list, deleting a protected branch, or changing the default branch if that branch is included in the security policy. | +| `prevent_approval_by_author` | `boolean` | false | `true`, `false` | When enabled, two person approval is required on all MRs as merge request authors cannot approve their own MRs and merge them unilaterally. | +| `prevent_approval_by_commit_author` | `boolean` | false | `true`, `false` | When enabled, users who have contributed code to the MR are ineligible for approval, ensuring code committers cannot introduce vulnerabilities and approve code to merge. | +| `remove_approvals_with_new_commit` | `boolean` | false | `true`, `false` | If an MR receives all necessary approvals to merge, but then a new commit is added, new approvals are required. This ensures new commits that may include vulnerabilities cannot be introduced. | +| `require_password_to_approve` | `boolean` | false | `true`, `false` | Password confirmation on approvals provides an additional level of security. Enabling this enforces the setting on all projects targeted by this policy. | +| `prevent_force_pushing` | `boolean` | false | `true`, `false` | Prevent pushing and force pushing to a protected branch. | ## Example security scan result policies project @@ -257,28 +267,47 @@ You can use this example in the YAML mode of the [Scan Result Policy editor](#sc It corresponds to a single object from the previous example: ```yaml -- name: critical vulnerability CS approvals - description: critical severity level only for container scanning - enabled: true - rules: - - type: scan_finding - branches: - - main - scanners: - - container_scanning - vulnerabilities_allowed: 1 - severity_levels: - - critical - vulnerability_states: - - newly_detected - actions: - - type: require_approval - approvals_required: 1 - user_approvers: - - adalberto.dare +type: scan_result_policy +name: critical vulnerability CS approvals +description: critical severity level only for container scanning +enabled: true +rules: +- type: scan_finding + branches: + - main + scanners: + - container_scanning + vulnerabilities_allowed: 1 + severity_levels: + - critical + vulnerability_states: + - newly_detected +actions: +- type: require_approval + approvals_required: 1 + user_approvers: + - adalberto.dare ``` -## Example situations where scan result policies require additional approval +## Understanding scan result policy approvals + +### Scope of scan result policy comparison + +- To determine when approval is required on a merge request, we compare the latest completed pipelines for each supported pipeline source for the source and target branch (for example, `feature`/`main`). This ensures the most comprehensive evaluation of scan results. +- We compare findings from the latest completed pipelines that ran on `HEAD` of the source and target branch. +- Scan result policies considers all supported pipeline sources (based on the [`CI_PIPELINE_SOURCE` variable](../../../ci/variables/predefined_variables.md)) when comparing results from both the source and target branches when determining if a merge request requires approval. Pipeline sources `webide` and `parent_pipeline` are not supported. + +### Accepting risk and ignoring vulnerabilities in future merge requests + +For scan result policies that are scoped to `newly_detected` findings, it's important to understand the implications of this vulnerability state. A finding is considered `newly_detected` if it exists on the merge request's branch but not on the default branch. When a merge request whose branch contains `newly_detected` findings is approved and merged, approvers are "accepting the risk" of those vulnerabilities. If one or more of the same vulnerabilities were detected after this time, their status would be `previously_detected` and so not be out of scope of a policy aimed at `newly_detected` findings. For example: + +- A scan result policy is created to block critical SAST findings. If a SAST finding for CVE-1234 is approved, future merge requests with the same violation will not require approval in the project. + +When using license approval policies, the combination of project, component (dependency), and license are considered in the evaluation. If a license is approved as an exception, future merge requests don't require approval for the same combination of project, component (dependency), and license. The component's version is not be considered in this case. If a previously approved package is updated to a new version, approvers will not need to re-approve. For example: + +- A license approval policy is created to block merge requests with newly detected licenses matching `AGPL-1.0`. A change is made in project `demo` for component `osframework` that violates the policy. If approved and merged, future merge requests to `osframework` in project `demo` with the license `AGPL-1.0` don't require approval. + +### Multiple approvals There are several situations where the scan result policy requires an additional approval step. For example: @@ -295,3 +324,43 @@ There are several situations where the scan result policy requires an additional - Someone stops a pipeline security job, and users can't skip the security scan. - A job in a merge request fails and is configured with `allow_failure: false`. As a result, the pipeline is in a blocked state. - A pipeline has a manual job that must run successfully for the entire pipeline to pass. + +### Known issues + +We have identified in [epic 11020](https://gitlab.com/groups/gitlab-org/-/epics/11020) common areas of confusion in scan result findings that need to be addressed. Below are a few of the known issues: + +- When using `newly_detected`, some findings may require approval when they are not introduced by the merge request (such as a new CVE on a related dependency). We currently use `main tip` of the target branch for comparison. In the future, we plan to use `merge base` for `newly_detected` policies (see [issue 428518](https://gitlab.com/gitlab-org/gitlab/-/issues/428518)). +- Findings or errors that cause approval to be required on a scan result policy may not be evident in the Security MR Widget. By using `merge base` in [issue 428518](https://gitlab.com/gitlab-org/gitlab/-/issues/428518) some cases will be addressed. We will additionally be [displaying more granular details](https://gitlab.com/groups/gitlab-org/-/epics/11185) about what caused security policy violations. +- Security policy violations are distinct compared to findings displayed in the MR widgets. Some violations may not be present in the MR widget. We are working to harmonize our features in [epic 11020](https://gitlab.com/groups/gitlab-org/-/epics/11020) and to display policy violations explicitly in merge requests in [epic 11185](https://gitlab.com/groups/gitlab-org/-/epics/11185). + +## Troubleshooting + +### Merge request rules widget shows a scan result policy is invalid or duplicated **(ULTIMATE SELF)** + +On GitLab self-managed from 15.0 to 16.4, the most likely cause is that the project was exported from a +group and imported into another, and had scan result policy rules. These rules are stored in a +separate project to the one that was exported. As a result, the project contains policy rules that +reference entities that don't exist in the imported project's group. The result is policy rules that +are invalid, duplicated, or both. + +To remove all invalid scan result policy rules from a GitLab instance, an administrator can run +the following script in the [Rails console](../../../administration/operations/rails_console.md). + +```ruby +Project.joins(:approval_rules).where(approval_rules: { report_type: %i[scan_finding license_scanning] }).where.not(approval_rules: { security_orchestration_policy_configuration_id: nil }).find_in_batches.flat_map do |batch| + batch.map do |project| + # Get projects and their configuration_ids for applicable project rules + [project, project.approval_rules.where(report_type: %i[scan_finding license_scanning]).pluck(:security_orchestration_policy_configuration_id).uniq] + end.uniq.map do |project, configuration_ids| # We take only unique combinations of project + configuration_ids + # If we find more configurations than what is available for the project, we take records with the extra configurations + [project, configuration_ids - project.all_security_orchestration_policy_configurations.pluck(:id)] + end.select { |_project, configuration_ids| configuration_ids.any? } +end.each do |project, configuration_ids| + # For each found pair project + ghost configuration, we remove these rules for a given project + Security::OrchestrationPolicyConfiguration.where(id: configuration_ids).each do |configuration| + configuration.delete_scan_finding_rules_for_project(project.id) + end + # Ensure we sync any potential rules from new group's policy + Security::ScanResultPolicies::SyncProjectWorker.perform_async(project.id) +end +``` diff --git a/doc/user/application_security/sast/customize_rulesets.md b/doc/user/application_security/sast/customize_rulesets.md index 90731114303..ed3b33fc35b 100644 --- a/doc/user/application_security/sast/customize_rulesets.md +++ b/doc/user/application_security/sast/customize_rulesets.md @@ -597,7 +597,7 @@ rules: The following example [enables SAST](index.md#configure-sast-in-your-cicd-yaml) and uses a shared ruleset customization file. The file is: -- Downloaded from a private project that requires authentication, by using a [Group Access Token](../../group/settings/group_access_tokens.md). +- Downloaded from a private project that requires authentication, by using a [Group Access Token](../../group/settings/group_access_tokens.md) securely stored within a CI variable. - Checked out at a specific Git commit SHA instead of the default branch. See [group access tokens](../../group/settings/group_access_tokens.md#bot-users-for-groups) for how to find the username associated with a group token. @@ -607,5 +607,5 @@ include: - template: Security/SAST.gitlab-ci.yml variables: - SAST_RULESET_GIT_REFERENCE: "group_2504721_bot_7c9311ffb83f2850e794d478ccee36f5:glpat-1234567@gitlab.com/example-group/example-ruleset-project@c8ea7e3ff126987fb4819cc35f2310755511c2ab" + SAST_RULESET_GIT_REFERENCE: "group_2504721_bot_7c9311ffb83f2850e794d478ccee36f5:$PERSONAL_ACCESS_TOKEN@gitlab.com/example-group/example-ruleset-project@c8ea7e3ff126987fb4819cc35f2310755511c2ab" ``` diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index acc7e9d9e84..770e24d87ca 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -273,6 +273,10 @@ When downloading, you always receive the most recent SAST artifact available. You can enable and configure SAST by using the UI, either with the default settings or with customizations. The method you can use depends on your GitLab license tier. +### Running jobs in merge request pipelines + +See [Use security scanning tools with merge request pipelines](../index.md#use-security-scanning-tools-with-merge-request-pipelines) + #### Configure SAST with customizations **(ULTIMATE ALL)** > [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/410013) individual SAST analyzers configuration options from the UI in GitLab 16.2. diff --git a/doc/user/application_security/sast/rules.md b/doc/user/application_security/sast/rules.md index e4054764e1f..3fb24bcd66b 100644 --- a/doc/user/application_security/sast/rules.md +++ b/doc/user/application_security/sast/rules.md @@ -102,7 +102,7 @@ More details are available in release announcements and in the CHANGELOG links p Key changes to the GitLab-managed ruleset for Semgrep-based scanning include: -- Beginning in GitLab 16.3, the GitLab Static Analysis and Vulnerability Research teams are working to remove rules that tend to produce too many false positive results or not enough actionable true positive results. Existing findings from these removed rules are [automatically resolved](index.md#automatic-vulnerability-resolution); they no longer appear in the [Security Dashboard](../security_dashboard/index.md#view-vulnerabilities-over-time-for-a-project) or in the default view of the [Vulnerability Report](../vulnerability_report/index.md). This work is tracked in [epic 10907](https://gitlab.com/groups/gitlab-org/-/epics/10907). +- Beginning in GitLab 16.3, the GitLab Static Analysis and Vulnerability Research teams are working to remove rules that tend to produce too many false positive results or not enough actionable true positive results. Existing findings from these removed rules are [automatically resolved](index.md#automatic-vulnerability-resolution); they no longer appear in the [Security Dashboard](../security_dashboard/index.md#project-security-dashboard) or in the default view of the [Vulnerability Report](../vulnerability_report/index.md). This work is tracked in [epic 10907](https://gitlab.com/groups/gitlab-org/-/epics/10907). - In GitLab 16.0 through 16.2, the GitLab Vulnerability Research team updated the guidance that's included in each result. - In GitLab 15.10, the `detect-object-injection` rule was [removed by default](https://gitlab.com/gitlab-org/gitlab/-/issues/373920) and its findings were [automatically resolved](index.md#automatic-vulnerability-resolution). diff --git a/doc/user/application_security/sast/troubleshooting.md b/doc/user/application_security/sast/troubleshooting.md index 34a2a3d01af..77a2f20c934 100644 --- a/doc/user/application_security/sast/troubleshooting.md +++ b/doc/user/application_security/sast/troubleshooting.md @@ -56,14 +56,14 @@ For information on this, see the [general Application Security troubleshooting s For information on this, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). -## Limitation when using rules:exists +## SAST jobs are running unexpectedly The [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) -uses the `rules:exists` parameter. For performance reasons, a maximum number of matches are made -against the given glob pattern. If the number of matches exceeds the maximum, the `rules:exists` +uses the `rules:exists` parameter. For performance reasons, a maximum number of 10000 matches are +made against the given glob pattern. If the number of matches exceeds the maximum, the `rules:exists` parameter returns `true`. Depending on the number of files in your repository, a SAST job might be -triggered even if the scanner doesn't support your project. For more details about this issue, see -the [`rules:exists` documentation](../../../ci/yaml/index.md#rulesexists). +triggered even if the scanner doesn't support your project. For more details about this limitation, +see the [`rules:exists` documentation](../../../ci/yaml/index.md#rulesexists). ## SpotBugs UTF-8 unmappable character errors diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 18016f6f342..4332b91c0f9 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -6,19 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Secret Detection **(FREE ALL)** -> - In GitLab 13.1, Secret Detection was split from the [SAST configuration](../sast/index.md#configuration) -> into its own CI/CD template. If you're using GitLab 13.0 or earlier and SAST is enabled, then -> Secret Detection is already enabled. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/222788) from GitLab Ultimate to GitLab -> Free in 13.3. -> - [In GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/297269), Secret Detection jobs -> `secret_detection_default_branch` and `secret_detection` were consolidated into one job, -> `secret_detection`. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an interactive reading and how-to demo of this Secret Detection doc, see [How to enable secret detection in GitLab Application Security Part 1/2](https://youtu.be/dbMxeO6nJCE?feature=shared) and [How to enable secret detection in GitLab Application Security Part 2/2](https://youtu.be/VL-_hdiTazo?feature=shared) -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) +> [In GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/297269), Secret Detection jobs `secret_detection_default_branch` and `secret_detection` were consolidated into one job, `secret_detection`. People sometimes accidentally commit secrets like keys or API tokens to Git repositories. After a sensitive value is pushed to a remote repository, anyone with access to the repository can impersonate the authorized user of the secret for malicious purposes. @@ -37,6 +25,13 @@ With GitLab Ultimate, Secret Detection results are also processed so you can: - Review them in the security dashboard. - [Automatically respond](automatic_response.md) to leaks in public repositories. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an interactive reading and how-to demo of this Secret Detection documentation see: + +- [How to enable secret detection in GitLab Application Security Part 1/2](https://youtu.be/dbMxeO6nJCE?feature=shared) +- [How to enable secret detection in GitLab Application Security Part 2/2](https://youtu.be/VL-_hdiTazo?feature=shared) + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For other interactive reading and how-to demos, see the [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9). + ## Detected secrets GitLab maintains the detection rules used in Secret Detection. @@ -111,26 +106,13 @@ Secret Detection can detect if a secret was added in one commit and removed in a - Merge request In a merge request, Secret Detection scans every commit made on the source branch. To use this - feature, you must use the [`latest` Secret Detection template](#templates), as it supports + feature, you must use the [`latest` Secret Detection template](../index.md#use-security-scanning-tools-with-merge-request-pipelines), as it supports [merge request pipelines](../../../ci/pipelines/merge_request_pipelines.md). Secret Detection's results are only available after the pipeline is completed. -## Templates +## Running jobs in merge request pipelines -Secret Detection default configuration is defined in CI/CD templates. Updates to the template are -provided with GitLab upgrades, allowing you to benefit from any improvements and additions. - -Available templates: - -- [`Secret-Detection.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.gitlab-ci.yml): Stable, default version of the Secret Detection CI/CD template. -- [`Secret-Detection.latest.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.latest.gitlab-ci.yml): Latest version of the Secret Detection template. - -WARNING: -The latest version of the template may include breaking changes. Use the stable template unless you -need a feature provided only in the latest template. - -For more information about template versioning, see the -[CI/CD documentation](../../../development/cicd/templates.md#latest-version). +See [Use security scanning tools with merge request pipelines](../index.md#use-security-scanning-tools-with-merge-request-pipelines) ## Enable Secret Detection @@ -166,7 +148,7 @@ your GitLab CI/CD configuration file is complex. ```yaml include: - - template: Security/Secret-Detection.gitlab-ci.yml + - template: Jobs/Secret-Detection.gitlab-ci.yml ``` 1. Select the **Validate** tab, then select **Validate pipeline**. @@ -232,7 +214,7 @@ This example uses a specific minor version of the analyzer: ```yaml include: - - template: Security/Secret-Detection.gitlab-ci.yml + - template: Jobs/Secret-Detection.gitlab-ci.yml secret_detection: variables: @@ -262,7 +244,7 @@ In the following example _extract_ of a `.gitlab-ci.yml` file: ```yaml include: - - template: Security/Secret-Detection.gitlab-ci.yml + - template: Jobs/Secret-Detection.gitlab-ci.yml secret_detection: variables: @@ -322,7 +304,7 @@ variables: SECRET_DETECTION_IMAGE_SUFFIX: '-fips' include: - - template: Security/Secret-Detection.gitlab-ci.yml + - template: Jobs/Secret-Detection.gitlab-ci.yml ``` ## Full history Secret Detection @@ -576,7 +558,7 @@ Prerequisites: ```yaml include: - - template: Security/Secret-Detection.gitlab-ci.yml + - template: Jobs/Secret-Detection.gitlab-ci.yml variables: SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard.png Binary files differnew file mode 100644 index 00000000000..1d324b8207a --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard.png Binary files differnew file mode 100644 index 00000000000..46fdebca9cd --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard.png diff --git a/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png b/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png Binary files differdeleted file mode 100644 index c2780fce787..00000000000 --- a/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 53a6dfe6d0a..89c950f2473 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -7,64 +7,42 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Security Dashboards and Security Center **(ULTIMATE ALL)** -You can use Security Dashboards to view trends about vulnerabilities -detected by [security scanners](../index.md#application-coverage). -These trends are shown in projects, groups, and the Security Center. +## Security Dashboards -To use the Security Dashboards, you must: +Security Dashboards are used to assess the security posture of your applications. GitLab provides +you with a collection of metrics, ratings, and charts for the vulnerabilities detected by the [security scanners](../index.md#application-coverage) run on your project. The security dashboard provides data such as: -- Configure at least one [security scanner](../index.md#application-coverage) in a project. -- Configure jobs to use the [`reports` syntax](../../../ci/yaml/index.md#artifactsreports). -- Use [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or later. If you use the - shared runners on GitLab.com, you are using the correct version. -- Have the [correct role](../../permissions.md) for the project or group. +- Vulnerability trends over a 30, 60, or 90-day time-frame for all projects in a group +- A letter grade rating for each project based on vulnerability severity +- The total number of vulnerabilities detected within the last 365 days including their severity + +The data provided by the Security Dashboards can be used supply to insight on what decisions can be made to improve your security posture. For example, using the 365 day trend view, you can see on which days a significant number of vulnerabilities were introduced. Then you can examine the code changes performed on those particular days in order perform a root-cause analysis to create better policies for preventing the introduction of vulnerabilities in the future. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Security Dashboard](https://www.youtube.com/watch?v=QHQHN4luNpc). -## When Security Dashboards are updated - -The Security Dashboards show results of scans from the most recent completed pipeline on the -[default branch](../../project/repository/branches/default.md). -Dashboards are updated with the result of completed pipelines run on the default branch; they do not include vulnerabilities discovered in pipelines from other un-merged branches. - -If you use manual jobs, for example gate deployments, in the default branch's pipeline, -the results of any scans are only updated when the job has been successfully run. -If manual jobs are skipped regularly, you should to define the job as optional, -using the [`allow_failure`](../../../ci/jobs/job_control.md#types-of-manual-jobs) attribute. - -To ensure regular security scans (even on infrequently developed projects), -you should use [scan execution policies](../../../user/application_security/policies/scan-execution-policies.md). -Alternatively, you can -[configure a scheduled pipeline](../../../ci/pipelines/schedules.md). +## Prerequisites -## Reduce false negatives in dependency scans +To view the Security Dashboards, the following is required: -WARNING: -False negatives occur when you resolve dependency versions during a scan, which differ from those -resolved when your project built and released in a previous pipeline. +- [Maintainer Role](../../permissions.md#roles) for the project or group. +- At least one [security scanner](../index.md#application-coverage) configured within your project. +- A successful security scan performed on the [default branch](../../project/repository/branches/default.md) of your project -To reduce false negatives in [dependency scans](../../../user/application_security/dependency_scanning/index.md) in scheduled pipelines, ensure you: - -- Include a lock file in your project. A lock file lists all transient dependencies and tracks their versions. - - Java projects can't have lock files. - - Python projects can have lock files, but GitLab Secure tools don't support them. -- Configure your project for [Continuous Delivery](../../../ci/introduction/index.md). +**Note**: +The Security Dashboards show results of scans from the most recent completed pipeline on the +[default branch](../../project/repository/branches/default.md). Dashboards are updated with the result of completed pipelines run on the default branch; they do not include vulnerabilities discovered in pipelines from other un-merged branches. -## View vulnerabilities over time for a project +## Viewing the Security Dashboard -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235558) in GitLab 13.6. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285476) in GitLab 13.10, options to zoom in on a date range, and download the vulnerabilities chart. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285477) in GitLab 13.11, date range slider to visualize data between given dates. +The Security Dashboard can be seen at the project, group, and the Security Center levels. +Each dashboard provides a unique viewpoint of your security posture. -The project Security Dashboard shows the total number of vulnerabilities -over time, with up to 365 days of historical data. Data refresh begins daily at 01:15 UTC via a scheduled job. -Each refresh captures a snapshot of open vulnerabilities. Data is not backported to prior days -so vulnerabilities opened after the job has already run for the day cannot be reflected in the -counts until the following day's refresh job. -Project Security Dashboards show statistics for all vulnerabilities with a current status of `Needs triage` or `Confirmed` . +### Project Security Dashboard -To view total number of vulnerabilities over time: +The Project Security Dashboard shows the total number of vulnerabilities detected over time, +with up to 365 days of historical data for a given project. You can view the Project Security +Dashboard: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Secure > Security dashboard**. @@ -75,70 +53,63 @@ To view total number of vulnerabilities over time: across the chart. - To reset to the original range, select **Remove Selection** (**{redo}**). -### Download the vulnerabilities chart + -To download an SVG image of the vulnerabilities chart: +#### Downloading the vulnerability chart + +You can download an image of the vulnerability chart from the Project Security Dashboard +to use in documentation, presentations, and so on. To download the image of the vulnerability +chart: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Secure > Security dashboard**. 1. Select **Save chart as an image** (**{download}**). -## View vulnerabilities over time for a group - -The group Security Dashboard gives an overview of vulnerabilities found in the default -branches of projects in a group and its subgroups. - -To view vulnerabilities over time for a group: - -1. On the left sidebar, select **Search or go to** and find your group. -1. Select **Security > Security dashboard**. -1. Hover over the chart to get more details about vulnerabilities. - - You can display the vulnerability trends over a 30, 60, or 90-day time frame (the default is 90 days). - - To view aggregated data beyond a 90-day time frame, use the - [VulnerabilitiesCountByDay GraphQL API](../../../api/graphql/reference/index.md#vulnerabilitiescountbyday). - GitLab retains the data for 365 days. - -## View project security status for a group - -Use the group Security Dashboard to view the security status of projects. - -To view project security status for a group: +You will then be prompted to download the image in SVG format. -1. On the left sidebar, select **Search or go to** and find your group. -1. Select **Secure > Security dashboard**. - -Each project is assigned a letter [grade](#project-vulnerability-grades) according to the highest-severity open vulnerability. -Dismissed or resolved vulnerabilities are excluded. Each project can receive only one letter grade and appears only once -in the Project security status report. +### Group Security Dashboard -To view vulnerabilities, go to the group's [vulnerability report](../vulnerability_report/index.md). +The group Security Dashboard provides an overview of vulnerabilities found in the default +branches of all projects in a group and its subgroups. The Group Security Dashboard +supplies the following: -### Project vulnerability grades +- Vulnerability trends over a 30, 60, or 90-day time frame +- A letter grade for each project in the group according to its highest-severity open vulnerability. The letter grades are assigned using the following criteria: | Grade | Description | -| --- | --- | +| ----- | ----------- | | **F** | One or more `critical` vulnerabilities | | **D** | One or more `high` or `unknown` vulnerabilities | | **C** | One or more `medium` vulnerabilities | | **B** | One or more `low` vulnerabilities | | **A** | Zero vulnerabilities | -## Security Center +To view group security dashboard: -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3426) in GitLab 13.4. +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Security > Security dashboard**. +1. Hover over the **Vulnerabilities over time** chart to get more details about vulnerabilities. + - You can display the vulnerability trends over a 30, 60, or 90-day time frame (the default is 90 days). + - To view aggregated data beyond a 90-day time frame, use the [VulnerabilitiesCountByDay GraphQL API](../../../api/graphql/reference/index.md#vulnerabilitiescountbyday). GitLab retains the data for 365 days. -The Security Center is a personal space where you view vulnerabilities across all your projects. It -shows the vulnerabilities present in the default branches of the projects. +1. Select the arrows under the **Project security status** section to see the what projects fall under a particular letter-grade rating: + - You can see how many vulnerabilities of a particular severity are found in a project + - You can select a project's name to directly access its project security dashboard -The Security Center includes: + -- The group Security Dashboard. -- A [vulnerability report](../vulnerability_report/index.md). -- A settings area to configure which projects to display. +## Security Center - +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3426) in GitLab 13.4. + +The Security Center is a configurable personal space where you can view vulnerabilities across all the +projects you belong to. The Security Center includes: -### View the Security Center +- The group Security Dashboard +- A [vulnerability report](../vulnerability_report/index.md) +- A settings area to configure which projects to display + +### Viewing the Security Center To view the Security Center: @@ -146,7 +117,9 @@ To view the Security Center: 1. Select **Your work**. 1. Select **Security > Security dashboard**. -### Add projects to the Security Center +The Security Center is blank by default. You must add a project which have been configured with at least one security scanner. + +### Adding Projects to the Security Center To add projects to the Security Center: @@ -157,26 +130,9 @@ To add projects to the Security Center: 1. Use the **Search your projects** text box to search for and select projects. 1. Select **Add projects**. -After you add projects, the security dashboard and vulnerability report show the vulnerabilities -found in those projects' default branches. - -You can add a maximum of 1,000 projects, however the **Project** filter in the **Vulnerability -Report** is limited to 100 projects. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +After you add projects, the security dashboard and vulnerability report show the vulnerabilities found in those projects' default branches. You can add a maximum of 1,000 projects, however the **Project** filter in the **Vulnerability Report** is limited to 100 projects. ## Related topics -- [Address the vulnerabilities](../vulnerabilities/index.md) - [Vulnerability reports](../vulnerability_report/index.md) - [Vulnerability Page](../vulnerabilities/index.md) diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index 0f0a61a2b02..f09672685de 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -259,7 +259,7 @@ A finding's primary identifier is a value that is unique to each finding. The ex of the finding's [first identifier](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/v2.4.0-rc1/dist/sast-report-format.json#L228) combine to create the value. -Examples of primary identifiers include `PluginID` for OWASP Zed Attack Proxy (ZAP), or `CVE` for +Examples of primary identifiers include `PluginID` for Zed Attack Proxy (ZAP), or `CVE` for Trivy. The identifier must be stable. Subsequent scans must return the same value for the same finding, even if the location has slightly changed. diff --git a/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4.png b/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4.png Binary files differdeleted file mode 100644 index 55694fc7926..00000000000 --- a/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4.png +++ /dev/null diff --git a/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4_updated.png b/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4_updated.png Binary files differnew file mode 100644 index 00000000000..7c1a5d4e298 --- /dev/null +++ b/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4_updated.png diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 34c57292767..476b2411621 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -104,7 +104,13 @@ When dismissing a vulnerability, one of the following reasons must be chosen to - **Used in tests**: The finding is not a vulnerability because it is part of a test or is test data. - **Not applicable**: The vulnerability is known, and has not been remediated or mitigated, but is considered to be in a part of the application that will not be updated. -## Change status of a vulnerability +## Change the status of a vulnerability + +> In GitLab 16.4 the ability for `Developers` to change the status of a vulnerability (`admin_vulnerability`) was [deprecated](../../../update/deprecations.md#deprecate-change-vulnerability-status-from-the-developer-role). The `admin_vulnerability` permission will be removed, by default, from all `Developer` roles in GitLab 17.0. + +Prerequisites: + +- You must have at least the Developer role for the project. To change a vulnerability's status from its Vulnerability Page: @@ -146,8 +152,9 @@ The issue is then opened so you can take further action. Prerequisites: -- [Enable Jira integration](../../../integration/jira/index.md). The **Enable Jira issue creation - from vulnerabilities** option must be selected as part of the configuration. +- [Enable Jira integration](../../../integration/jira/configure.md). The + **Enable Jira issue creation from vulnerabilities** option must be selected as part + of the configuration. - Each user must have a personal Jira user account with permission to create issues in the target project. @@ -242,7 +249,7 @@ To resolve a vulnerability, you can either: - [Resolve a vulnerability with a merge request](#resolve-a-vulnerability-with-a-merge-request). - [Resolve a vulnerability manually](#resolve-a-vulnerability-manually). - + ### Resolve a vulnerability with a merge request diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 24ed318e688..e71aab5839e 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -11,7 +11,8 @@ The Vulnerability Report provides information about vulnerabilities from scans o cumulative results of all successful jobs, regardless of whether the pipeline was successful. The scan results from a pipeline are only ingested after all the jobs in the pipeline complete. -The report is available for users with the [correct role](../../permissions.md) on projects, groups, and the Security Center. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Vulnerability Management](https://www.youtube.com/watch?v=8SJHz6BCgXM). At all levels, the Vulnerability Report contains: @@ -19,8 +20,11 @@ At all levels, the Vulnerability Report contains: - Filters for common vulnerability attributes. - Details of each vulnerability, presented in tabular layout. -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Vulnerability Management](https://www.youtube.com/watch?v=8SJHz6BCgXM). +At the project level, the Vulnerability Report also contains: + +- A time stamp showing when it was updated, including a link to the latest pipeline. +- The number of failures that occurred in the most recent pipeline. Select the failure + notification to view the **Failed jobs** tab of the pipeline's page. The **Activity** column contains icons to indicate the activity, if any, taken on the vulnerability in that row: @@ -38,56 +42,38 @@ status of a Jira issue is not shown in the GitLab UI.  -## Project-level Vulnerability Report - -At the project level, the Vulnerability Report also contains: - -- A time stamp showing when it was updated, including a link to the latest pipeline. -- The number of failures that occurred in the most recent pipeline. Select the failure - notification to view the **Failed jobs** tab of the pipeline's page. - When vulnerabilities originate from a multi-project pipeline setup, this page displays the vulnerabilities that originate from the selected project. -### View the project-level vulnerability report +## View the vulnerability report -To view the project-level vulnerability report: +View the vulnerability report to list all vulnerabilities in the project or group. -1. On the left sidebar, select **Search or go to** and find your project. -1. Select **Secure > Vulnerability report**. +Prerequisites: -## Vulnerability Report actions +- You must have at least the Developer role for the project or group. -From the Vulnerability Report you can: +To view the vulnerability report: -- [Filter the list of vulnerabilities](#filter-the-list-of-vulnerabilities). -- [View more details about a vulnerability](#view-details-of-a-vulnerability). -- [View vulnerable source location](#view-vulnerable-source-location) (if available). -- [Change the status of vulnerabilities](#change-status-of-vulnerabilities). -- [Export details of vulnerabilities](#export-vulnerability-details). -- [Sort vulnerabilities by date](#sort-vulnerabilities-by-date-detected). -- [Manually add a vulnerability finding](#manually-add-a-vulnerability-finding). -- [Grouping vulnerability report](#group-vulnerabilities) +1. On the left sidebar, select **Search or go to** and find your project or group. +1. Select **Secure > Vulnerability report**. ## Vulnerability Report filters You can filter the Vulnerability Report to narrow focus on only vulnerabilities matching specific criteria. -The available filters are: +The filters available at all levels are: <!-- vale gitlab.SubstitutionWarning = NO --> -- **Status**: Detected, Confirmed, Dismissed, Resolved. For details on what each status means, see +- **Status**: Detected, confirmed, dismissed, resolved. For details on what each status means, see [vulnerability status values](../vulnerabilities/index.md#vulnerability-status-values). -- **Severity**: Critical, High, Medium, Low, Info, Unknown. +- **Severity**: Critical, high, medium, low, info, unknown. - **Tool**: For more details, see [Tool filter](#tool-filter). -- **Project**: For more details, see [Project filter](#project-filter). - **Activity**: For more details, see [Activity filter](#activity-filter). -The filters' criteria are combined to show only vulnerabilities matching all criteria. -An exception to this behavior is the Activity filter. For more details about how it works, see -[Activity filter](#activity-filter). +Additionally, the [project filter](#project-filter) is available at the group level. <!-- vale gitlab.SubstitutionWarning = YES --> @@ -106,8 +92,6 @@ After each filter is selected: ### Tool filter -> The third-party tool filter was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229661) in GitLab 13.12. - The tool filter allows you to focus on vulnerabilities detected by selected tools. When using the tool filter, you can choose: @@ -122,23 +106,28 @@ For details of each of the available tools, see [Security scanning tools](../ind The content of the Project filter depends on the current level: -- **Security Center**: Only projects you've [added to your personal Security Center](../security_dashboard/index.md#add-projects-to-the-security-center). +- **Security Center**: Only projects you've [added to your personal Security Center](../security_dashboard/index.md#adding-projects-to-the-security-center). - **Group level**: All projects in the group. - **Project level**: Not applicable. ### Activity filter -The Activity filter behaves differently from the other filters. The selected values form mutually -exclusive sets to allow for precisely locating the desired vulnerability records. Additionally, not -all options can be selected in combination. +The activity filter behaves differently from the other filters. You can select only one value in +each category. -Selection behavior when using the Activity filter: +Selection behavior when using the activity filter: -- **All**: Vulnerabilities with any Activity status (same as ignoring this filter). Selecting this deselects any other Activity filter options. -- **No activity**: Only vulnerabilities without either an associated issue or that are no longer detected. Selecting this deselects any other Activity filter options. -- **With issues**: Only vulnerabilities with one or more associated issues. Does not include vulnerabilities that also are no longer detected. -- **No longer detected**: Only vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. Does not include vulnerabilities with one or more associated issues. -- **With issues** and **No longer detected**: Only vulnerabilities that have one or more associated issues and also are no longer detected in the latest pipeline scan of the `default` branch. +- **Activity** + - **All activity**: Vulnerabilities with any activity status (same as ignoring this filter). Selecting this deselects all other activity filter options. +- **Detection** + - **Still detected**: Vulnerabilities that are still detected in the latest pipeline scan of the `default` branch. + - **No longer detected**: Vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. +- **Issue** + - **Has issues**: Vulnerabilities with one or more associated issues. + - **Does not have issue**: Vulnerabilities without an associated issue. +- **Merge request** + - **Has merge request**: Vulnerabilities with one or more associated merge requests. + - **Does not have merge request**: Vulnerabilities without an associated merge request. ## View details of a vulnerability @@ -186,7 +175,7 @@ Fields included are: - Group name - Project name -- Scanner type +- Tool - Scanner name - Status - Vulnerability @@ -200,6 +189,8 @@ Fields included are: - Location - Activity: Returns `true` if the vulnerability is resolved on the default branch, and `false` if not. - Comments +- Full Path +- CVSS Vectors NOTE: Full details are available through our @@ -259,8 +250,8 @@ Group, Project, and Security Center Vulnerability Reports. To filter them, use t ## Group vulnerabilities -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420055) in GitLab 16.4. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/422509) in GitLab 16.5. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420055) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `vulnerability_report_grouping`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/422509) in GitLab 16.6. Feature flag `vulnerability_report_grouping` removed. To group the Vulnerability Report: diff --git a/doc/user/clusters/agent/gitops/example_repository_structure.md b/doc/user/clusters/agent/gitops/example_repository_structure.md index 02eea3300af..52855b9731c 100644 --- a/doc/user/clusters/agent/gitops/example_repository_structure.md +++ b/doc/user/clusters/agent/gitops/example_repository_structure.md @@ -96,7 +96,7 @@ You've successfully created a repository with a protected deployment branch! Next, you'll configure CI/CD to merge changes from the default branch to your deployment branch. -In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/yaml/gitlab_ci_yaml.md) file with the following contents: +In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/index.md#the-gitlab-ciyml-file) file with the following contents: ```yaml deploy: diff --git a/doc/user/clusters/agent/gitops/flux_oci_tutorial.md b/doc/user/clusters/agent/gitops/flux_oci_tutorial.md index b970c818a72..2c4796adf2b 100644 --- a/doc/user/clusters/agent/gitops/flux_oci_tutorial.md +++ b/doc/user/clusters/agent/gitops/flux_oci_tutorial.md @@ -65,7 +65,7 @@ First, create a repository for your Kubernetes manifests: Next, configure [GitLab CI/CD](../../../../ci/index.md) to package your manifests into an OCI artifact, and push the artifact to the [GitLab Container Registry](../../../packages/container_registry/index.md): -1. In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/yaml/gitlab_ci_yaml.md) file with the following contents: +1. In the root of `web-app-manifests`, create and push a [`.gitlab-ci.yml`](../../../../ci/index.md#the-gitlab-ciyml-file) file with the following contents: ```yaml package: diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md index 27724a95291..832f91691e8 100644 --- a/doc/user/clusters/agent/gitops/flux_tutorial.md +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -121,6 +121,7 @@ To install `agentk`: kind: Secret metadata: name: gitlab-agent-token + namespace: gitlab type: Opaque stringData: token: "<your-token-here>" diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index d620a9f658c..588be3a1223 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -76,7 +76,7 @@ In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `ce Prerequisites: - For a [GitLab CI/CD workflow](../ci_cd_workflow.md), ensure that - [GitLab CI/CD is not disabled](../../../../ci/enable_or_disable_ci.md#disable-cicd-in-a-project). + [GitLab CI/CD is not disabled](../../../../ci/pipelines/settings.md#disable-gitlab-cicd-pipelines). You must register an agent before you can install the agent in your cluster. To register an agent: @@ -220,7 +220,7 @@ The following example projects can help you get started with the agent. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, GitLab warns you on the agent's list page to update the agent version installed on your cluster. -For the best experience, the version of the agent installed in your cluster should match the GitLab major and minor version. The previous minor version is also supported. For example, if your GitLab version is v14.9.4 (major version 14, minor version 9), then versions v14.9.0 and v14.9.1 of the agent are ideal, but any v14.8.x version of the agent is also supported. See [the release page](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/releases) of the GitLab agent. +For the best experience, the version of the agent installed in your cluster should match the GitLab major and minor version. The previous and next minor versions are also supported. For example, if your GitLab version is v14.9.4 (major version 14, minor version 9), then versions v14.9.0 and v14.9.1 of the agent are ideal, but any v14.8.x or v14.10.x version of the agent is also supported. See [the release page](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/releases) of the GitLab agent. ### Update the agent version diff --git a/doc/user/clusters/agent/user_access.md b/doc/user/clusters/agent/user_access.md index 21dc249b1d1..b3735770a97 100644 --- a/doc/user/clusters/agent/user_access.md +++ b/doc/user/clusters/agent/user_access.md @@ -151,15 +151,66 @@ Prerequisite: - You have an agent configured with the `user_access` entry. -To grant Kubernetes API access: +### Configure local access with the GitLab CLI (recommended) + +You can use the [GitLab CLI `glab`](../../../editor_extensions/gitlab_cli/index.md) to create or update +a Kubernetes configuration file to access the agent Kubernetes API. + +Use `glab cluster agent` commands to manage cluster connections: + +1. View a list of all the agents associated with your project: + +```shell +glab cluster agent list --repo '<group>/<project>' + +# If your current working directory is the Git repository of the project with the agent, you can omit the --repo option: +glab cluster agent list +``` + +1. Use the numerical agent ID presented in the first column of the output to update your `kubeconfig`: + +```shell +glab cluster agent update-kubeconfig --repo '<group>/<project>' --agent '<agent-id>' --use-context +``` + +1. Verify the update with `kubectl` or your preferred Kubernetes tooling: + +```shell +kubectl get nodes +``` + +The `update-kubeconfig` command sets `glab cluster agent get-token` as a +[credential plugin](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins) +for Kubernetes tools to retrieve a token. The `get-token` command creates and +returns a personal access token that is valid until the end of the current day. +Kubernetes tools cache the token until it expires, the API returns an authorization error, or the process exits. Expect all subsequent calls to your Kubernetes tooling to create a new token. + +The `glab cluster agent update-kubeconfig` command supports a number of command line flags. You can view all supported flags with `glab cluster agent update-kubeconfig --help`. + +Some examples: + +```shell +# When the current working directory is the Git repository where the agent is registered the --repo / -R flag can be omitted +glab cluster agent update-kubeconfig --agent '<agent-id>' + +# When the --use-context option is specified the `current-context` of the kubeconfig file is changed to the agent context +glab cluster agent update-kubeconfig --agent '<agent-id>' --use-context + +# The --kubeconfig flag can be used to specify an alternative kubeconfig path +glab cluster agent update-kubeconfig --agent '<agent-id>' --kubeconfig ~/gitlab.kubeconfig +``` + +### Configure local access manually using a personal access token + +You can configure access to a Kubernetes cluster using a long-lived personal access token: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Operate > Kubernetes clusters** and retrieve the numerical ID of the agent you want to access. You need the ID to construct the full API token. 1. Create a [personal access token](../../profile/personal_access_tokens.md) with the `k8s_proxy` scope. You need the access token to construct the full API token. -1. Construct `kube config` entries to access the cluster: - 1. Make sure that the proper `kube config` is selected. +1. Construct `kubeconfig` entries to access the cluster: + 1. Make sure that the proper `kubeconfig` is selected. For example, you can set the `KUBECONFIG` environment variable. - 1. Add the GitLab KAS proxy cluster to the `kube config`: + 1. Add the GitLab KAS proxy cluster to the `kubeconfig`: ```shell kubectl config set-cluster <cluster_name> --server "https://kas.gitlab.com/k8s-proxy" diff --git a/doc/user/clusters/agent/vulnerabilities.md b/doc/user/clusters/agent/vulnerabilities.md index a2dc50e43d7..e57551fc8c1 100644 --- a/doc/user/clusters/agent/vulnerabilities.md +++ b/doc/user/clusters/agent/vulnerabilities.md @@ -20,7 +20,7 @@ If both `agent config` and `scan execution policies` are configured, the configu ### Enable via agent configuration -To enable scanning of all images within your Kubernetes cluster via the agent configuration, add a `container_scanning` configuration block to your agent +To enable scanning of images within your Kubernetes cluster via the agent configuration, add a `container_scanning` configuration block to your agent configuration with a `cadence` field containing a [CRON expression](https://en.wikipedia.org/wiki/Cron) for when the scans are run. ```yaml @@ -39,9 +39,9 @@ Other elements of the [CRON syntax](https://docs.oracle.com/cd/E12058_01/doc/doc NOTE: The CRON expression is evaluated in [UTC](https://www.timeanddate.com/worldclock/timezone/utc) using the system-time of the Kubernetes-agent pod. -By default, operational container scanning attempts to scan the workloads in all -namespaces for vulnerabilities. You can set the `vulnerability_report` block with the `namespaces` -field which can be used to restrict which namespaces are scanned. For example, +By default, operational container scanning does not scan any workloads for vulnerabilities. +You can set the `vulnerability_report` block with the `namespaces` +field which can be used to select which namespaces are scanned. For example, if you would like to scan only the `default`, `kube-system` namespaces, you can use this configuration: ```yaml @@ -112,13 +112,15 @@ You can customize it with a `resource_requirements` field. container_scanning: resource_requirements: requests: - cpu: 200m + cpu: '0.2' memory: 200Mi limits: - cpu: 700m + cpu: '0.7' memory: 700Mi ``` +When using a fractional value for CPU, format the value as a string. + NOTE: Resource requirements can only be set up using the agent configuration. If you enabled `Operational Container Scanning` through `scan execution policies`, you would need to define the resource requirements within the agent configuration file. @@ -143,3 +145,10 @@ You must have at least the Developer role. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/415451) in GitLab 16.4. To scan private images, the scanner relies on the image pull secrets (direct references and from the service account) to pull the image. + +## Troubleshooting + +### `Error running Trivy scan. Container terminated reason: OOMKilled` + +OCS might fail with an OOM error if there are too many resources to be scanned or if the images being scanned are large. +To resolve this, [configure the resource requirement](#configure-scanner-resource-requirements) to increase the amount of memory available. diff --git a/doc/user/compliance/compliance_center/index.md b/doc/user/compliance/compliance_center/index.md index 0e205a29920..4a42a70a7e7 100644 --- a/doc/user/compliance/compliance_center/index.md +++ b/doc/user/compliance/compliance_center/index.md @@ -111,9 +111,9 @@ You can sort the compliance report on: You can filter the compliance violations report on: -- Project. -- Date range of merge. -- Target branch. +- The project that the violation was found on. +- The date range of violation. +- The target branch of the violation. Select a row to see details of the compliance violation. @@ -393,6 +393,7 @@ On self-managed GitLab, by default this feature is not available. To make it ava With compliance frameworks report, you can see all the compliance frameworks in a group. Each row of the report shows: - Framework name. +- Associated projects. The default framework for the group has a **default** badge. diff --git a/doc/user/compliance/license_list.md b/doc/user/compliance/license_list.md index f315f319b71..7ad19775509 100644 --- a/doc/user/compliance/license_list.md +++ b/doc/user/compliance/license_list.md @@ -16,7 +16,7 @@ For the licenses to appear under the license list, the following requirements must be met: 1. You must be generating an SBOM file with components from one of our [one of our supported languages](license_scanning_of_cyclonedx_files/index.md#supported-languages-and-package-managers). -1. If using our [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/License-Scanning.gitlab-ci.yml) to generate the SBOM file, then your project must use at least one of the [supported languages and package managers](license_scanning_of_cyclonedx_files/index.md#supported-languages-and-package-managers). +1. If using our [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) to generate the SBOM file, then your project must use at least one of the [supported languages and package managers](license_scanning_of_cyclonedx_files/index.md#supported-languages-and-package-managers). Alternatively, licenses will also appear under the license list when using our deprecated [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/License-Scanning.gitlab-ci.yml) as long as the following requirements are met: diff --git a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md index 81f7cc61782..5d7a689e610 100644 --- a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md +++ b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md @@ -22,16 +22,11 @@ Licenses not in the SPDX list are reported as "Unknown". License information can ## Configuration -Prerequisites: +To enable License scanning of CycloneDX files: -- On GitLab self-managed only, enable [Synchronization with the GitLab License Database](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. On GitLab SaaS this step has already been completed. - Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#enabling-the-analyzer) and ensure that its prerequisites are met. - -From the `.gitlab-ci.yml` file, remove the deprecated line `Jobs/License-Scanning.gitlab-ci.yml`, if -it's present. - -On GitLab self-managed only, you can [choose package registry metadata to sync](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. +- On GitLab self-managed only, you can [choose package registry metadata to synchronize](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. For this data synchronization to work, you must allow outbound network traffic from your GitLab instance to the domain `storage.googleapis.com`. If you have limited or no network connectivity then please refer to the documentation section [running in an offline environment](#running-in-an-offline-environment) for further guidance. ## Supported languages and package managers diff --git a/doc/user/custom_roles.md b/doc/user/custom_roles.md index a13c45306ad..bbb48724078 100644 --- a/doc/user/custom_roles.md +++ b/doc/user/custom_roles.md @@ -13,35 +13,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Ability to view a vulnerability report [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123835) in GitLab 16.1. > - [Feature flag `custom_roles_vulnerability` removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124049) in GitLab 16.2. > - Ability to create and remove a custom role with the UI [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393235) in GitLab 16.4. -> - Ability to manage group members [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17364) in GitLab 16.5 under `admin_group_member` Feature flag. -> - Ability to manage project access tokens [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/421778) in GitLab 16.5 under `manage_project_access_tokens` Feature flag. +> - Ability to manage group members [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17364) in GitLab 16.5. +> - Ability to manage project access tokens [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/421778) in GitLab 16.5 [with a flag](../administration/feature_flags.md) named `manage_project_access_tokens`. +> - Ability to archive projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/425957) in GitLab 16.6 in [with a flag](../administration/feature_flags.md) named `archive_project`. Disabled by default. -Custom roles allow group members who are assigned the Owner role to create roles +Custom roles allow group Owners or instance administrators to create roles specific to the needs of their organization. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a demo of the custom roles feature, see [[Demo] Ultimate Guest can view code on private repositories via custom role](https://www.youtube.com/watch?v=46cp_-Rtxps). -The following granular permissions are available. You can add these permissions to any base role, and add them in combination with each other to create a customized role: - -- The Guest+1 role, which allows users with the Guest role to view code. -- In GitLab 16.1 and later, you can create a custom role that can view vulnerability reports and change the status of the vulnerabilities. -- In GitLab 16.3 and later, you can create a custom role that can view the dependency list. -- In GitLab 16.4 and later, you can create a custom role that can approve merge requests. -- In GitLab 16.5 and later, you can create a custom role that can manage group members. - You can discuss individual custom role and permission requests in [issue 391760](https://gitlab.com/gitlab-org/gitlab/-/issues/391760). -When you enable a custom role for a user with the Guest role, that user has -access to elevated permissions, and therefore: - -- Is considered a [billable user](../subscriptions/self_managed/index.md#billable-users) on self-managed GitLab. -- [Uses a seat](../subscriptions/gitlab_com/index.md#how-seat-usage-is-determined) on GitLab.com. - -This does not apply to Guest+1, a Guest custom role that only enables the `read_code` -permission. Users with that specific custom role are not considered billable users -and do not use a seat. - ## Create a custom role Prerequisites: @@ -51,9 +34,19 @@ Prerequisites: - The group must be in the Ultimate tier. - You must have: - At least one private project so that you can see the effect of giving a - user with the Guest role a custom role. The project can be in the group itself + user a custom role. The project can be in the group itself or one of that group's subgroups. - - A [personal access token with the API scope](profile/personal_access_tokens.md#create-a-personal-access-token). + - If you are using the API to create the custom role, a [personal access token with the API scope](profile/personal_access_tokens.md#create-a-personal-access-token). + +You create a custom role by selecting [permissions](#available-permissions) to add +to a base role. + +You can select any number of permissions. For example, you can create a custom role +with the ability to: + +- View vulnerability reports. +- Change the status of vulnerabilities. +- Approve merge requests. ### GitLab SaaS @@ -64,7 +57,7 @@ Prerequisite: 1. On the left sidebar, select **Search or go to** and find your group. 1. Select **Settings > Roles and Permissions**. 1. Select **Add new role**. -1. In **Base role to use as template**, select **Guest**. +1. In **Base role to use as template**, select an existing non-custom role. 1. In **Role name**, enter the custom role's title. 1. Select the **Permissions** for the new custom role. 1. Select **Create new role**. @@ -80,30 +73,44 @@ Prerequisite: 1. Select **Settings > Roles and Permissions**. 1. From the top dropdown list, select the group you want to create a custom role in. 1. Select **Add new role**. -1. In **Base role to use as template**, select **Guest**. +1. In **Base role to use as template**, select an existing non-custom role. 1. In **Role name**, enter the custom role's title. 1. Select the **Permissions** for the new custom role. 1. Select **Create new role**. To create a custom role, you can also [use the API](../api/member_roles.md#add-a-member-role-to-a-group). -### Custom role requirements +### Available permissions + +The following permissions are available. You can add these permissions in any combination +to a base role to create a custom role. + +Some permissions require having other permissions enabled first. For example, administration of vulnerabilities (`admin_vulnerability`) can only be enabled if reading vulnerabilities (`read_vulnerability`) is also enabled. + +These requirements are documented in the `Required permission` column in the following table. -For every ability, a minimal access level is defined. To be able to create a custom role which enables a certain ability, the `member_roles` table record has to have the associated minimal access level. For all abilities, the minimal access level is Guest. Only users who have at least the Guest role can be assigned to a custom role. +| Permission | Version | Required permission | Description | +| ------------------------------- | -----------------------| -------------------- | ----------- | +| `read_code` | GitLab 15.7 and later | Not applicable | View project code. Does not include the ability to pull code. | +| `read_vulnerability` | GitLab 16.1 and later | Not applicable | View [vulnerability reports](application_security/vulnerability_report/index.md). | +| `admin_vulnerability` | GitLab 16.1 and later | `read_vulnerability` | Change the [status of vulnerabilities](application_security/vulnerabilities/index.md#vulnerability-status-values). | +| `read_dependency` | GitLab 16.3 and later | Not applicable | View [project dependencies](application_security/dependency_list/index.md). | +| `admin_merge_request` | GitLab 16.4 and later | Not applicable | View and approve [merge requests](project/merge_requests/index.md), and view the associated merge request code. <br> Does not allow users to view or change merge request approval rules. | +| `manage_project_access_tokens` | GitLab 16.5 and later | Not applicable | Create, delete, and list [project access tokens](project/settings/project_access_tokens.md). | +| `admin_group_member` | GitLab 16.5 and later | Not applicable | Add or remove [group members](group/manage.md). | +| `archive_project` | GitLab 16.6 and later | Not applicable | Archive and unarchive [projects](project/settings/index.md#archive-a-project). | -Some roles and abilities require having other abilities enabled. For example, a custom role can only have administration of vulnerabilities (`admin_vulnerability`) enabled if reading vulnerabilities (`read_vulnerability`) is also enabled. +## Billing and seat usage -You can see the abilities requirements in the following table. +When you enable a custom role for a user with the Guest role, that user has +access to elevated permissions over the base role, and therefore: -| Ability | Required ability | -| -- | -- | -| `read_code` | - | -| `read_dependency` | - | -| `read_vulnerability` | - | -| `admin_merge_request` | - | -| `admin_vulnerability` | `read_vulnerability` | -| `admin_group_member` | - | -| `manage_project_access_tokens` | - | +- Is considered a [billable user](../subscriptions/self_managed/index.md#billable-users) on self-managed GitLab. +- [Uses a seat](../subscriptions/gitlab_com/index.md#how-seat-usage-is-determined) on GitLab.com. + +This does not apply when the user's custom role only has the `read_code` permission +enabled. Guest users with that specific permission only are not considered billable users +and do not use a seat. ## Associate a custom role with an existing group member @@ -147,14 +154,14 @@ To do this, you can either remove the custom role from all group members with th ### Remove a custom role from a group member To remove a custom role from a group member, use the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) -and pass an empty `member_role_id` value. +and pass an empty `member_role_id` value: ```shell # to update a project membership -curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" +curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": null, "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" # to update a group membership -curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" +curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": null, "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" ``` ### Remove a group member with a custom role from the group diff --git a/doc/user/discussions/img/add_internal_note_v15_0.png b/doc/user/discussions/img/add_internal_note_v15_0.png Binary files differdeleted file mode 100644 index cf052edd5e7..00000000000 --- a/doc/user/discussions/img/add_internal_note_v15_0.png +++ /dev/null diff --git a/doc/user/discussions/img/add_internal_note_v16_6.png b/doc/user/discussions/img/add_internal_note_v16_6.png Binary files differnew file mode 100644 index 00000000000..0d6b4c05160 --- /dev/null +++ b/doc/user/discussions/img/add_internal_note_v16_6.png diff --git a/doc/user/discussions/img/create_thread_v16_6.png b/doc/user/discussions/img/create_thread_v16_6.png Binary files differnew file mode 100644 index 00000000000..3e0abb3d589 --- /dev/null +++ b/doc/user/discussions/img/create_thread_v16_6.png diff --git a/doc/user/discussions/img/discussion_comment.png b/doc/user/discussions/img/discussion_comment.png Binary files differdeleted file mode 100644 index 3fec5962363..00000000000 --- a/doc/user/discussions/img/discussion_comment.png +++ /dev/null diff --git a/doc/user/discussions/img/quickly_assign_commenter_v13_1.png b/doc/user/discussions/img/quickly_assign_commenter_v13_1.png Binary files differdeleted file mode 100644 index aa8f65ef6c4..00000000000 --- a/doc/user/discussions/img/quickly_assign_commenter_v13_1.png +++ /dev/null diff --git a/doc/user/discussions/img/quickly_assign_commenter_v16_6.png b/doc/user/discussions/img/quickly_assign_commenter_v16_6.png Binary files differnew file mode 100644 index 00000000000..7d6e54fdfa2 --- /dev/null +++ b/doc/user/discussions/img/quickly_assign_commenter_v16_6.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index ae74b534e02..a3ed888ed53 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -156,12 +156,12 @@ Prerequisite: To lock an issue or merge request: -1. On the right sidebar, next to **Lock issue** or **Lock merge request**, select **Edit**. +1. On the right sidebar, next to **Lock discussion**, select **Edit**. 1. On the confirmation dialog, select **Lock**. Notes are added to the page details. -If an issue or merge request is locked and closed, you cannot reopen it. +If an issue or merge request is closed with a locked discussion, then you cannot reopen it until the discussion is unlocked. <!-- Delete when the `moved_mr_sidebar` feature flag is removed --> If you don't see this action on the right sidebar, your project or instance might have [moved sidebar actions](../project/merge_requests/index.md#move-sidebar-actions) enabled. @@ -192,7 +192,7 @@ To add an internal note: 1. Below the comment, select the **Make this an internal note** checkbox. 1. Select **Add internal note**. - + You can also mark an [issue as confidential](../project/issues/confidential_issues.md). @@ -233,7 +233,7 @@ You can assign an issue to a user who made a comment. 1. In the comment, select the **More Actions** (**{ellipsis_v}**) menu. 1. Select **Assign to commenting user**: -  +  1. To unassign the commenter, select the button again. ## Create a thread by replying to a standard comment @@ -272,9 +272,9 @@ To create a thread: 1. From the list, select **Start thread**. 1. Select **Start thread** again. -A threaded comment is created. + - +A threaded comment is created. ## Resolve a thread diff --git a/doc/user/feature_flags.md b/doc/user/feature_flags.md index f665395b103..88928ab6d47 100644 --- a/doc/user/feature_flags.md +++ b/doc/user/feature_flags.md @@ -1,6 +1,6 @@ --- stage: none -group: Development +group: unassigned info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" description: "View a list of all the flags available in the GitLab application." layout: 'feature_flags' diff --git a/doc/user/free_push_limit.md b/doc/user/free_push_limit.md index c0b23720ab1..c1be8287eb1 100644 --- a/doc/user/free_push_limit.md +++ b/doc/user/free_push_limit.md @@ -6,9 +6,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Free push limit **(FREE SAAS)** -A 100 MB per-file limit applies when pushing new files to any project in the Free tier. +A 100 MiB per-file limit applies when pushing new files to any project in the Free tier. -If a new file that is 100 MB or large is pushed to a project in the Free tier, an error is displayed. For example: +If a new file that is 100 MiB or large is pushed to a project in the Free tier, an error is displayed. For example: ```shell Enumerating objects: 3, done. diff --git a/doc/user/gitlab_duo_chat.md b/doc/user/gitlab_duo_chat.md new file mode 100644 index 00000000000..ba6cd9b8f21 --- /dev/null +++ b/doc/user/gitlab_duo_chat.md @@ -0,0 +1,67 @@ +--- +stage: AI-powered +group: Duo Chat +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference +--- + +# Answer questions with GitLab Duo Chat **(ULTIMATE SAAS EXPERIMENT)** + +> Introduced in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). + +You can get AI generated support from GitLab Duo Chat about the following topics: + +- How to use GitLab. +- Questions about an issue. +- Question about an epic. +- Questions about a code file. +- Follow-up questions to answers from the chat. + +Example questions you might ask: + +- `Explain the concept of a 'fork' in a concise manner.` +- `Provide step-by-step instructions on how to reset a user's password.` +- `Generate a summary for the issue identified via this link: <link to your issue>` +- `Generate a concise summary of the description of the current issue.` + +The examples above all use data from either the issue or the GitLab documentation. However, you can also ask to generate code, CI/CD configurations, or to explain code. For example: + +- `Write a Ruby function that prints 'Hello, World!' when called.` +- `Develop a JavaScript program that simulates a two-player Tic-Tac-Toe game. Provide both game logic and user interface, if applicable.` +- `Create a .gitlab-ci.yml configuration file for testing and building a Ruby on Rails application in a GitLab CI/CD pipeline.` +- `Provide a clear explanation of the given Ruby code: def sum(a, b) a + b end. Describe what this code does and how it works.` + +In addition to the provided prompts, feel free to ask follow-up questions to delve deeper into the topic or task at hand. This helps you get more detailed and precise responses tailored to your specific needs, whether it's for further clarification, elaboration, or additional assistance. + +- A follow-up to the question `Write a Ruby function that prints 'Hello, World!' when called.` could be: + - `Could you also explain how I can call and execute this Ruby function in a typical Ruby environment, such as the command line?` + +This is an experimental feature and we're continuously extending the capabilities and reliability of the chat. + +## Enable GitLab Duo Chat + +To use this feature, at least one group you're a member of must: + +- Have the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features) enabled. + +## Use GitLab Duo Chat + +1. In the lower-left corner, select the **Help** icon. + The [new left sidebar must be enabled](../tutorials/left_sidebar/index.md). +1. Select **GitLab Duo Chat**. A drawer opens on the right side of your screen. +1. Enter your question in the chat input box and press **Enter** or select **Send**. It may take a few seconds for the interactive AI chat to produce an answer. +1. You can ask a follow-up question. +1. If you want to ask a new question unrelated to the previous conversation, you may receive better answers if you clear the context by typing `/reset` into the input box and selecting **Send**. + +NOTE: +Only the last 50 messages are retained in the chat history. The chat history expires 3 days after last use. + +## Give Feedback + +Your feedback is important to us as we continually enhance your GitLab Duo Chat experience: + +- **Enhance Your Experience**: Leaving feedback helps us customize the Chat for your needs and improve its performance for everyone. +- **Privacy Assurance**: Rest assured, we don't collect your prompts. Your privacy is respected, and your interactions remain private. + +To give feedback about a specific response, use the feedback buttons in the response message. +Or, you can add a comment in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/415591). diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index 966945b6b12..53a62a60157 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -118,7 +118,7 @@ To allow runner downloading, add the [outbound runner CIDR ranges](../gitlab_com > - Support for restricting access to projects in the group [added](https://gitlab.com/gitlab-org/gitlab/-/issues/14004) in GitLab 14.1.2. > - Support for restricting group memberships to groups with a subset of the allowed email domains [added](https://gitlab.com/gitlab-org/gitlab/-/issues/354791) in GitLab 15.1.1 -You can prevent users with email addresses in specific domains from being added to a group and its projects. +You can prevent users with email addresses in specific domains from being added to a group and its projects. You can define an email domain allowlist at the top-level namespace only. Subgroups do not offer the ability to define an alternative allowlist. To restrict group access by domain: @@ -260,6 +260,13 @@ Group syncing allows LDAP groups to be mapped to GitLab groups. This provides mo Group links can be created by using either a CN or a filter. To create these group links, go to the group's **Settings > LDAP Synchronization** page. After configuring the link, it may take more than an hour for the users to sync with the GitLab group. +If a user is a member of two configured LDAP groups for the same GitLab group, they are granted the higher of the roles associated with the two LDAP groups. +For example: + +- User is a member of LDAP groups `Owner` and `Dev`. +- The GitLab Group is configured with these two LDAP groups. +- When group sync is completed, the user is granted the Owner role as this is the higher of the two LDAP group roles. + For more information on the administration of LDAP and group sync, refer to the [main LDAP documentation](../../administration/auth/ldap/ldap_synchronization.md#group-sync). NOTE: diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 5675393441e..a5cc3ad9070 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -206,7 +206,7 @@ To view epics in a group: Whether you can view an epic depends on the [group visibility level](../../public_access.md) and the epic's [confidentiality status](#make-an-epic-confidential): -- Public group and a non-confidential epic: You don't have to be a member of the group. +- Public group and a non-confidential epic: Anyone can view the epic. - Private group and non-confidential epic: You must have at least the Guest role for the group. - Confidential epic (regardless of group visibility): You must have at least the Reporter role for the group. diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index e1d5c8e5f0a..24d5ca5b214 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -240,7 +240,16 @@ To view group import history: 1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. 1. Select **Import group**. 1. In the upper-right corner, select **History**. -1. If there are any errors for a particular import, you can see them by selecting **Details**. +1. If there are any errors for a particular import, select **See failures** to see their details. + +### Review results of the import + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/429109) in GitLab 16.6 [with a flag](../../feature_flags.md) named `bulk_import_details_page`. Enabled by default. + +To review the results of an import: + +1. Go to the [Group import history page](#group-import-history). +1. To see the details of a failed import, select the **See failures** link on any import with a **Failed** status. ### Migrated group items @@ -337,7 +346,7 @@ Project items that are migrated to the destination GitLab instance include: | Projects | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) | | Auto DevOps | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339410) | | Badges | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75029) | -| Branches (including protected branches) | [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/339414) | +| Branches (including protected branches) <sup>1</sup> | [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/339414) | | CI Pipelines | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339407) | | Commit comments | [GitLab 15.10](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) | | Designs | [GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/339421) | @@ -361,6 +370,14 @@ Project items that are migrated to the destination GitLab instance include: | Uploads | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339401) | | Wikis | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/345923) | +<html> +<small>Footnotes: + <ol> + <li>Imported branches respect the [default branch protection settings](../../project/protected_branches.md) of the destination group, which can cause an unprotected branch to be imported as protected.</li> + </ol> +</small> +</html> + #### Issue-related items Issue-related project items that are migrated to the destination GitLab instance include: diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 484fd8c533b..1a4fa9df305 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -202,7 +202,7 @@ A table displays the member's: NOTE: The display of group members' **Source** might be inconsistent. -For more information, see [issue 414557](https://gitlab.com/gitlab-org/gitlab/-/issues/414557). +For more information, see [issue 23020](https://gitlab.com/gitlab-org/gitlab/-/issues/23020). ## Filter and sort members in a group @@ -219,7 +219,7 @@ Filter a group to find members. By default, all members in the group and subgrou In lists of group members, entries can display the following badges: - **SAML**, to indicate the member has a [SAML account](saml_sso/index.md) connected to them. -- **Enterprise**, to indicate that the member is an [enterprise user](../enterprise_user/index.md). +- **Enterprise**, to indicate that the member of the top-level group is an [enterprise user](../enterprise_user/index.md). 1. On the left sidebar, select **Search or go to** and find your group. 1. Select **Manage > Members**. @@ -227,7 +227,7 @@ In lists of group members, entries can display the following badges: - To view members in the group only, select **Membership = Direct**. - To view members of the group and its subgroups, select **Membership = Inherited**. - To view members with two-factor authentication enabled or disabled, select **2FA = Enabled** or **Disabled**. - - [In GitLab 14.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/349887), to view GitLab users created by [SAML SSO](saml_sso/index.md) or [SCIM provisioning](saml_sso/scim_setup.md) select **Enterprise = true**. + - To view members of the top-level group who are [enterprise users](../enterprise_user/index.md), select **Enterprise = true**. ### Search a group diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index d671b0434b6..48f86ee4f0e 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -130,6 +130,11 @@ After sharing the `Frontend` group with the `Engineering` group: - The **Groups** tab lists the `Engineering` group. - The **Groups** tab lists a group regardless of whether it is a public or private group. +- From [GitLab 16.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134623), + the invited group's name and membership source will be masked unless: + - the invited group is public, or + - the current user is a member of the invited group, or + - the current user is a member of the current group. - All direct members of the `Engineering` group have access to the `Frontend` group. The least access is granted between the access in the `Engineering` group and the access in the `Frontend` group. - If `Member1` has the Maintainer role in `Engineering` and `Engineering` is added to `Frontend` with the Developer role, `Member1` has the Developer role in `Frontend`. - If `Member2` has the Guest role in `Engineering` and `Engineering` is added to `Frontend` with the Developer role, `Member2` has the Guest role in `Frontend`. @@ -487,29 +492,6 @@ To enable Experiment features for a top-level group: 1. Under **Experiment and Beta features**, select the **Use Experiment and Beta features** checkbox. 1. Select **Save changes**. -## Enable third-party AI features **(ULTIMATE SAAS)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118222) in GitLab 16.0. - -WARNING: -These AI features use [third-party services](../ai_features.md#data-usage) -and require transmission of data, including personal data. - -All users in the group have third-party AI features enabled by default. -This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) -that belong to the group. - -To disable third-party AI features for a group: - -1. On the left sidebar, select **Search or go to** and find your group. -1. Select **Settings > General**. -1. Expand **Permissions and group features**. -1. Under **Third-party AI services**, uncheck the **Use third-party AI services** checkbox. -1. Select **Save changes**. - -When Code Suggestions are enabled and disabled, an -[audit event](../../administration/audit_events.md#view-audit-events) is created. - ## Group activity analytics **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/experiment-beta-support.md#beta). diff --git a/doc/user/group/reporting/git_abuse_rate_limit.md b/doc/user/group/reporting/git_abuse_rate_limit.md index 1b14edb04d9..d32524b8f5f 100644 --- a/doc/user/group/reporting/git_abuse_rate_limit.md +++ b/doc/user/group/reporting/git_abuse_rate_limit.md @@ -13,7 +13,7 @@ On self-managed GitLab, by default this feature is not available. To make it ava This is the group-level documentation. For self-managed instances, see the [administration documentation](../../admin_area/reporting/git_abuse_rate_limit.md). -Git abuse rate limiting is a feature to automatically ban users who download, clone, pull, fetch, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. +Git abuse rate limiting is a feature to automatically ban users who download, clone, pull, fetch, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with [personal](../../../user/profile/personal_access_tokens.md) or [group access tokens](../../../user/group/settings/group_access_tokens.md), as well as [CI/CD job tokens](../../../ci/jobs/ci_job_token.md). Access to unrelated groups is unaffected. Git abuse rate limiting does not apply to top-level group owners, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md index c18ccaf9c20..7b10da016b9 100644 --- a/doc/user/group/saml_sso/group_sync.md +++ b/doc/user/group/saml_sso/group_sync.md @@ -81,6 +81,8 @@ When SAML is enabled, users with the Maintainer or Owner role see a new menu item in group **Settings > SAML Group Links**. You can configure one or more **SAML Group Links** to map a SAML identity provider group name to a GitLab role. This can be done for a top-level group or any subgroup. +SAML Group Sync only manages a group if that group has one or more SAML group links. If a SAML group link is created then removed, the user remains in the group until they are removed from the group in the identity provider. + To link the SAML groups: 1. In **SAML Group Name**, enter the value of the relevant `saml:AttributeValue`. The value entered here must exactly match the value sent in the SAML response. For some IdPs, this may be a group ID or object ID (Azure AD) instead of a friendly group name. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 444afd3442b..70af800b180 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -54,7 +54,8 @@ To set up SSO with Azure as your identity provider: 1. You should set the following attributes: - **Unique User Identifier (Name identifier)** to `user.objectID`. - **nameid-format** to `persistent`. For more information, see how to [manage user SAML identity](#manage-user-saml-identity). - - **Additional claims** to [supported attributes](#user-attributes). + - **email** to `user.mail` or similar. + - **Additional claims** to [supported attributes](#configure-assertions). 1. Make sure the identity provider is set to have provider-initiated calls to link existing GitLab accounts. @@ -98,7 +99,7 @@ To set up Google Workspace as your identity provider: - For **Last name**: `last_name`. - For **Name ID format**: `EMAIL`. - For **NameID**: `Basic Information > Primary email`. - For more information, see [manage user SAML identity](#manage-user-saml-identity). + For more information, see [supported attributes](#configure-assertions). 1. Make sure the identity provider is set to have provider-initiated calls to link existing GitLab accounts. @@ -134,6 +135,8 @@ To set up SSO with Okta as your identity provider: 1. Set these values: - For **Application username (NameID)**: **Custom** `user.getInternalProperty("id")`. - For **Name ID Format**: `Persistent`. For more information, see [manage user SAML identity](#manage-user-saml-identity). + - For **email**: `user.email` or similar. + - For additional **Attribute Statements**, see [supported attributes](#configure-assertions). 1. Make sure the identity provider is set to have provider-initiated calls to link existing GitLab accounts. @@ -170,10 +173,28 @@ To set up OneLogin as your identity provider: | **Identity provider single sign-on URL** | **SAML 2.0 Endpoint** | 1. For **NameID**, use `OneLogin ID`. For more information, see [manage user SAML identity](#manage-user-saml-identity). - +1. Configure [required and supported attributes](#configure-assertions). 1. Make sure the identity provider is set to have provider-initiated calls to link existing GitLab accounts. +### Configure assertions + +At minimum, you must configure the following assertions: + +1. [NameID](#manage-user-saml-identity). +1. Email. + +Optionally, you can pass user information to GitLab as attributes in the SAML assertion. + +- The user's email address can be an **email** or **mail** attribute. +- The username can be either a **username** or **nickname** attribute. You should specify only + one of these. + +For more information, see the [attributes available for self-managed GitLab instances](../../../integration/saml.md#configure-assertions). + +NOTE: +Attribute names starting with phrases such as `http://schemas.microsoft.com/ws/2008/06/identity/claims/` are not supported. For more information on configuring required attribute names in the SAML identity provider's settings, see [example group SAML and SCIM configurations](../../../user/group/saml_sso/example_saml_config.md). + ### Use metadata To configure some identity providers, you need a GitLab metadata URL. @@ -253,19 +274,6 @@ When a user tries to sign in with Group SSO, GitLab attempts to find or create a - Create a new account with another email address. - Sign-in to their existing account to link the SAML identity. -### User attributes - -You can pass user information to GitLab as attributes in the SAML assertion. - -- The user's email address can be an **email** or **mail** attribute. -- The username can be either a **username** or **nickname** attribute. You should specify only - one of these. - -For more information, see the [attributes available for self-managed GitLab instances](../../../integration/saml.md#configure-assertions). - -NOTE: -Attribute names starting with phrases such as `http://schemas.microsoft.com/ws/2008/06/identity/claims/` are not supported. For more information on configuring required attribute names in the SAML identity provider's settings, see [example group SAML and SCIM configurations](../../../user/group/saml_sso/example_saml_config.md). - ### Link SAML to your existing GitLab.com account > **Remember me** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/121569) in GitLab 15.7. diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md index 9d3cc0bef50..527d710058a 100644 --- a/doc/user/group/saml_sso/troubleshooting.md +++ b/doc/user/group/saml_sso/troubleshooting.md @@ -222,7 +222,7 @@ to [reset their password](https://gitlab.com/users/password/new) if both: Users might get an error that states "SAML Name ID and email address do not match your user account. Contact an administrator." This means: -- The NameID value sent by SAML does not match the existing SAML identity `extern_uid` value. +- The NameID value sent by SAML does not match the existing SAML identity `extern_uid` value. Both the NameID and the `extern_uid` are case sensitive. For more information, see [manage user SAML identity](index.md#manage-user-saml-identity). - Either the SAML response did not include an email address or the email address did not match the user's GitLab email address. The workaround is that a GitLab group Owner uses the [SAML API](../../../api/saml.md) to update the user's SAML `extern_uid`. @@ -356,3 +356,21 @@ If you see this message after trying to invite a user to a group: 1. Ensure the user is a [member of the top-level group](../index.md#search-a-group). Additionally, see [troubleshooting users receiving a 404 after sign in](#users-receive-a-404). + +## Message: The SAML response did not contain an email address. Either the SAML identity provider is not configured to send the attribute, or the identity provider directory does not have an email address value for your user + +This error appears when the SAML response does not contain the user's email address in an **email** or **mail** attribute as shown in the following example: + +```xml +<Attribute Name="email"> + <AttributeValue>user@domain.com‹/AttributeValue> +</Attribute> +``` + +Attribute names starting with phrases such as `http://schemas.microsoft.com/ws/2008/06/identity/claims/` like in the following example are not supported. Remove this type of attribute name from the SAML response on the IDP side. + +```xml +<Attribute Name="http://schemas.microsoft.com/ws/2008/06/identity/claims/email"> + <AttributeValue>user@domain.com‹/AttributeValue> +</Attribute> +``` diff --git a/doc/user/group/saml_sso/troubleshooting_scim.md b/doc/user/group/saml_sso/troubleshooting_scim.md index 703dff16fd5..b31c2eed9df 100644 --- a/doc/user/group/saml_sso/troubleshooting_scim.md +++ b/doc/user/group/saml_sso/troubleshooting_scim.md @@ -4,7 +4,7 @@ group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Troubleshooting SCIM **(PREMIUM SAAS)** +# Troubleshooting SCIM **(FREE ALL)** This section contains possible solutions for problems you might encounter. @@ -31,6 +31,8 @@ To solve this problem: 1. Have the user sign in directly to GitLab. 1. [Manually link](scim_setup.md#link-scim-and-saml-identities) their account. +Alternatively, self-managed administrators can [add a user identity](../../../administration/admin_area.md#user-identities). + ## User cannot sign in The following are possible solutions for problems where users cannot sign in: @@ -38,10 +40,11 @@ The following are possible solutions for problems where users cannot sign in: - Ensure that the user was added to the SCIM app. - If you receive the `User is not linked to a SAML account` error, the user probably already exists in GitLab. Have the user follow the [Link SCIM and SAML identities](scim_setup.md#link-scim-and-saml-identities) instructions. + Alternatively, self-managed administrators can [add a user identity](../../../administration/admin_area.md#user-identities). - The **Identity** (`extern_uid`) value stored by GitLab is updated by SCIM whenever `id` or `externalId` changes. Users - cannot sign in unless the GitLab Identity (`extern_uid`) value matches the `NameId` sent by SAML. This value is also - used by SCIM to match users on the `id`, and is updated by SCIM whenever the `id` or `externalId` values change. -- The SCIM `id` and SCIM `externalId` must be configured to the same value as the SAML `NameId`. You can trace SAML responses + cannot sign in unless the GitLab identifier (`extern_uid`) of the sign-in method matches the ID sent by the provider, such as + the `NameId` sent by SAML. This value is also used by SCIM to match users on the `id`, and is updated by SCIM whenever the `id` or `externalId` values change. +- On GitLab.com, the SCIM `id` and SCIM `externalId` must be configured to the same value as the SAML `NameId`. You can trace SAML responses using [debugging tools](troubleshooting.md#saml-debugging-tools), and check any errors against the [SAML troubleshooting](troubleshooting.md) information. @@ -94,10 +97,12 @@ When the SCIM app changes: - Users can follow the instructions in the [Change the SAML app](index.md#change-the-identity-provider) section. - Administrators of the identity provider can: - 1. Remove users from the SCIM app, which unlinks all removed users. + 1. Remove users from the SCIM app, which: + - In GitLab.com, removes all removed users from the group. + - In GitLab self-managed, blocks users. 1. Turn on sync for the new SCIM app to [link existing users](scim_setup.md#link-scim-and-saml-identities). -## SCIM app returns `"User has already been taken","status":409` error +## SCIM app returns `"User has already been taken","status":409` error **(PREMIUM SAAS)** Changing the SAML or SCIM configuration or provider can cause the following problems: @@ -109,7 +114,7 @@ Changing the SAML or SCIM configuration or provider can cause the following prob the SCIM app. 1. Use the same SCIM API to update the SCIM `extern_uid` for the user on GitLab.com. -## Search Rails logs for SCIM requests +## Search Rails logs for SCIM requests **(PREMIUM SAAS)** GitLab.com administrators can search for SCIM requests in the `api_json.log` using the `pubsub-rails-inf-gprd-*` index in [Kibana](https://about.gitlab.com/handbook/support/workflows/kibana.html#using-kibana). Use the following filters based diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index df9986e32e7..2ed01a0ec05 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -125,14 +125,17 @@ To view when the data was most recently updated, in the right corner next to **E ### How value stream analytics measures stages Value stream analytics measures each stage from its start event to its end event. +Only items that have reached their end event are included in the stage time calculation. -For example, a stage might start when a user adds a label to an issue, and ends when they add another label. -Items aren't included in the stage time calculation if they have not reached the end event. +By default, blocked issues are not included in the life cycle overview. +However, you can use custom labels (for example `workflow::blocked`) to track them. -Value stream analytics allows you to customize your stages based on pre-defined events. To make the -configuration easier, GitLab provides a pre-defined list of stages that can be used as a template +You can customize stages in value stream analytics based on pre-defined events. +To help you with the configuration, GitLab provides a pre-defined list of stages that you can use as a template. +For example, you can define a stage that starts when you add a label to an issue, +and ends when you add another label. -Each pre-defined stages of value stream analytics is further described in the table below. +The following table gives an overview of the pre-defined stages in value stream analytics. | Stage | Measurement method | | ------- | -------------------- | @@ -156,7 +159,7 @@ If a stage does not include a start and a stop time, its data is not included in In this example, milestones have been created and CI/CD for testing and setting environments is configured. - 09:00: Create issue. **Issue** stage starts. -- 11:00: Add issue to a milestone, start work on the issue, and create a branch locally. +- 11:00: Add issue to a milestone (or backlog), start work on the issue, and create a branch locally. **Issue** stage stops and **Plan** stage starts. - 12:00: Make the first commit. - 12:30: Make the second commit to the branch that mentions the issue number. diff --git a/doc/user/img/snippet_clone_button_v13_0.png b/doc/user/img/snippet_clone_button_v13_0.png Binary files differdeleted file mode 100644 index bf681e7349b..00000000000 --- a/doc/user/img/snippet_clone_button_v13_0.png +++ /dev/null diff --git a/doc/user/img/snippet_intro_v13_11.png b/doc/user/img/snippet_intro_v13_11.png Binary files differdeleted file mode 100644 index 4b6818341b7..00000000000 --- a/doc/user/img/snippet_intro_v13_11.png +++ /dev/null diff --git a/doc/user/img/snippet_sample_v16_6.png b/doc/user/img/snippet_sample_v16_6.png Binary files differnew file mode 100644 index 00000000000..035947a2b82 --- /dev/null +++ b/doc/user/img/snippet_sample_v16_6.png diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index 96819860a2f..5412ced3e6d 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -95,7 +95,7 @@ Use CI/CD environment variables to configure your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. 1. Set the variable `BASE64_GOOGLE_CREDENTIALS` to the `base64` encoded JSON file you just created. -1. Set the variable `TF_VAR_gcp_project` to your GCP `project` name. +1. Set the variable `TF_VAR_gcp_project` to your GCP `project` ID. 1. Set the variable `TF_VAR_agent_token` to the agent token displayed in the previous task. 1. Set the variable `TF_VAR_kas_address` to the agent server address displayed in the previous task. @@ -113,6 +113,10 @@ contains other variables that you can override according to your needs: Refer to the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. +## Enable Kubernetes Engine API + +From the Google Cloud console, enable the [Kubernetes Engine API](https://console.cloud.google.com/apis/library/container.googleapis.com). + ## Provision your cluster After configuring your project, manually trigger the provisioning of your cluster. In GitLab: diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 1e6c59c2253..65ec84652ef 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -85,7 +85,6 @@ To use a Terraform template: ```yaml variables: TF_STATE_NAME: default - TF_CACHE_KEY: default # If your terraform files are in a subdirectory, set TF_ROOT accordingly. For example: # TF_ROOT: terraform/production ``` diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index 24ae3c998f8..8fe639bb453 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -16,10 +16,13 @@ enabling you to see statistics about the resources that Terraform creates, modifies, or destroys. WARNING: -Like any other job artifact, Terraform Plan data is viewable by anyone with the Guest role for the repository. -Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform Plan -includes sensitive data such as passwords, access tokens, or certificates, we strongly -recommend encrypting plan output or modifying the project visibility settings. +Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. +Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform `plan.json` or `plan.cache` +files include sensitive data like passwords, access tokens, or certificates, you should +encrypt the plan output or modify the project visibility settings. You should also **disable** +[public pipelines](../../../ci/pipelines/settings.md#change-pipeline-visibility-for-non-project-members-in-public-projects) +and set the [artifact's public flag to false](../../../ci/yaml/index.md#artifactspublic) (`public: false`). +This setting ensures artifacts are accessible only to GitLab administrators and project members with at least the Reporter role. ## Configure Terraform report artifacts diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 081e20b158e..876300a7794 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -54,12 +54,12 @@ Prerequisites: WARNING: Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. -Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan -includes sensitive data, like passwords, access tokens, or certificates, you should -encrypt plan output or modify the project visibility settings. We also strongly recommend that you **disable** +Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform `plan.json` or `plan.cache` +files include sensitive data like passwords, access tokens, or certificates, you should +encrypt the plan output or modify the project visibility settings. You should also **disable** [public pipelines](../../../ci/pipelines/settings.md#change-pipeline-visibility-for-non-project-members-in-public-projects) -by setting the artifact's public flag to false (`public: false`). This setting ensures artifacts are -accessible only to GitLab Administrators and project members with the Reporter role and above. +and set the [artifact's public flag to false](../../../ci/yaml/index.md#artifactspublic) (`public: false`). +This setting ensures artifacts are accessible only to GitLab administrators and project members with at least the Reporter role. To configure GitLab CI/CD as a backend: diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 7f097891e92..a06e26c3e82 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -379,7 +379,8 @@ the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activati To prevent malicious activity, GitLab renders only the first 50 inline math instances. The number of math blocks is also limited based on render time. If the limit is exceeded, -GitLab renders the excess math instances as text. +GitLab renders the excess math instances as text. Wiki and repository files do not have +these limits. Math written between dollar signs with backticks (``$`...`$``) or single dollar signs (`$...$`) is rendered inline with the text. diff --git a/doc/user/okrs.md b/doc/user/okrs.md index 46390cd0275..ca5882da22a 100644 --- a/doc/user/okrs.md +++ b/doc/user/okrs.md @@ -399,6 +399,24 @@ To turn off a check-in reminder, enter: /checkin_reminder never ``` +## Set an objective as a parent + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/11198) in GitLab 16.6. + +Prerequisite: + +- You must have at least the Reporter role for the project. +- The parent objective and child OKR must belong to the same project. + +To set an objective as a parent of an OKR: + +1. [Open the objective](#view-an-objective) or [key result](#view-a-key-result) that you want to edit. +1. Next to **Parent**, from the dropdown list, select the parent to add. +1. Select any area outside the dropdown list. + +To remove the parent of the objective or key result, +next to **Parent**, select the dropdown list and then select **Unassign**. + ## Confidential OKRs > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8410) in GitLab 15.3. diff --git a/doc/user/organization/index.md b/doc/user/organization/index.md index 2a33543fea5..5a08307cc11 100644 --- a/doc/user/organization/index.md +++ b/doc/user/organization/index.md @@ -6,6 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Organization +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409913) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ui_for_organizations`. Disabled by default. + +FLAG: +This feature is not ready for production use. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `ui_for_organizations`. +On GitLab.com, this feature is not available. + DISCLAIMER: This page contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. @@ -37,6 +44,37 @@ see [epic 9265](https://gitlab.com/groups/gitlab-org/-/epics/9265). For a video introduction to the new hierarchy concept for groups and projects for epics, see [Consolidating groups and projects update (August 2021)](https://www.youtube.com/watch?v=fE74lsG_8yM). +## View organizations + +To view the organizations you have access to: + +- On the left sidebar, select **Organizations** (**{organization}**). + +## Create an organization + +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New organization**. +1. In the **Organization name** field, enter a name for the organization. +1. In the **Organization URL** field, enter a path for the organization. +1. Select **Create organization**. + +## Edit an organization's name + +1. On the left sidebar, select **Organizations** (**{organization}**) and find the organization you want to edit. +1. Select **Settings > General**. +1. Update the **Organization name** field. +1. Select **Save changes**. + +## Manage groups and projects + +1. On the left sidebar, select **Organizations** (**{organization}**) and find the organization you want to manage. +1. Select **Manage > Groups and projects**. +1. To switch between groups and projects, use the **Display** filter next to the search box. + +## Manage users + +1. On the left sidebar, select **Organizations** (**{organization}**) and find the organization you want to manage. +1. Select **Manage > Users**. + ## Related topics - [Organization developer documentation](../../development/organization/index.md) diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index d8662ef6512..6eac299e71f 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -225,7 +225,7 @@ To install a package: Using a CI/CD job token: ```shell - composer config gitlab-token.<DOMAIN-NAME> gitlab-ci-token ${CI_JOB_TOKEN} + composer config -- gitlab-token.<DOMAIN-NAME> gitlab-ci-token "${CI_JOB_TOKEN}" ``` Result in the `auth.json` file: diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 1f95d2f9403..786fd0ca658 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -79,7 +79,7 @@ For more information on running container images, see the [Docker documentation] Your container images must follow this naming convention: ```plaintext -<registry URL>/<namespace>/<project>/<image> +<registry server>/<namespace>/<project>[/<optional path>] ``` For example, if your project is `gitlab.example.com/mynamespace/myproject`, diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index 2af16dcc85a..8c4f25af2e1 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -15,14 +15,61 @@ if you add a large number of images or tags: You should delete unnecessary images and tags and set up a [cleanup policy](#cleanup-policy) to automatically manage your container registry usage. -## Check Container Registry storage use +## Check Container Registry storage use **(FREE SAAS)** The Usage Quotas page (**Settings > Usage Quotas > Storage**) displays storage usage for Packages. -This page includes the [Container Registry usage](../../usage_quotas.md#container-registry-usage), which is only available on GitLab.com. Measuring usage is only possible on the new version of the GitLab Container Registry backed by a metadata database, which is [available on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/5523) since GitLab 15.7. For information on the planned availability for self-managed instances, see [epic 5521](https://gitlab.com/groups/gitlab-org/-/epics/5521). +## How container registry usage is calculated + +Image layers stored in the Container Registry are deduplicated at the root namespace level. + +An image is only counted once if: + +- You tag the same image more than once in the same repository. +- You tag the same image across distinct repositories under the same root namespace. + +An image layer is only counted once if: + +- You share the image layer across multiple images in the same container repository, project, or group. +- You share the image layer across different repositories. + +Only layers that are referenced by tagged images are accounted for. Untagged images and any layers +referenced exclusively by them are subject to [online garbage collection](../container_registry/delete_container_registry_images.md#garbage-collection). +Untagged image layers are automatically deleted after 24 hours if they remain unreferenced during that period. + +Image layers are stored on the storage backend in the original (usually compressed) format. This +means that the measured size for any given image layer should match the size displayed on the +corresponding [image manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md#example-image-manifest). + +Namespace usage is refreshed a few minutes after a tag is pushed or deleted from any container repository under the namespace. + +### Delayed refresh + +It is not possible to calculate container registry usage +with maximum precision in real time for extremely large namespaces (about 1% of namespaces). +To enable maintainers of these namespaces to see their usage, there is a delayed fallback mechanism. +See [epic 9413](https://gitlab.com/groups/gitlab-org/-/epics/9413) for more details. + +If the usage for a namespace cannot be calculated with precision, GitLab falls back to the delayed method. +In the delayed method, the displayed usage size is the sum of **all** unique image layers +in the namespace. Untagged image layers are not ignored. As a result, +the displayed usage size might not change significantly after deleting tags. Instead, +the size value only changes when: + +- An automated [garbage collection process](../container_registry/delete_container_registry_images.md#garbage-collection) + runs and deletes untagged image layers. After a user deletes a tag, a garbage collection run + is scheduled to start 24 hours later. During that run, images that were previously tagged + are analyzed and their layers deleted if not referenced by any other tagged image. + If any layers are deleted, the namespace usage is updated. +- The namespace's registry usage shrinks enough that GitLab can measure it with maximum precision. + As usage for namespaces shrinks to be under the [limits](../../../user/usage_quotas.md#namespace-storage-limit), + the measurement switches automatically from delayed to precise usage measurement. + There is no place in the UI to determine which measurement method is being used, + but [issue 386468](https://gitlab.com/gitlab-org/gitlab/-/issues/386468) proposes to improve this. + ## Cleanup policy > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/218737) from "expiration policy" to "cleanup policy" in GitLab 13.2. diff --git a/doc/user/packages/container_registry/troubleshoot_container_registry.md b/doc/user/packages/container_registry/troubleshoot_container_registry.md index 13e14dfdeb4..3fb2754eb9c 100644 --- a/doc/user/packages/container_registry/troubleshoot_container_registry.md +++ b/doc/user/packages/container_registry/troubleshoot_container_registry.md @@ -128,6 +128,12 @@ time is set to 15 minutes. If you are using self-managed GitLab, an administrator can [increase the token duration](../../../administration/packages/container_registry.md#increase-token-duration). +## `Failed to pull image` messages + +You might receive a [`Failed to pull image'](../../../ci/debugging.md#failed-to-pull-image-messages) +error message when a CI/CD job is unable to pull a container image from a project with a limited +[CI/CD job token scope](../../../ci/jobs/ci_job_token.md#limit-job-token-scope-for-public-or-internal-projects). + ## Slow uploads when using `kaniko` to push large images When you push large images with `kaniko`, you might experience uncharacteristically long delays. @@ -136,3 +142,24 @@ This is typically a result of [a performance issue with `kaniko` and HTTP/2](htt The current workaround is to use HTTP/1.1 when pushing with `kaniko`. To use HTTP/1.1, set the `GODEBUG` environment variable to `"http2client=0"`. + +## `docker login` command fails with `access forbidden` + +The container registry [returns the GitLab API URL to the Docker client](../../../administration/packages/container_registry.md#architecture-of-gitlab-container-registry) +to validate credentials. The Docker client uses basic auth, so the request contains +the `Authorization` header. If the `Authorization` header is missing in the request to the +`/jwt/auth` endpoint configured in the `token_realm` for the registry configuration, +you receive an `access forbidden` error message. + +For example: + +```plaintext +> docker login gitlab.example.com:4567 + +Username: user +Password: +Error response from daemon: Get "https://gitlab.company.com:4567/v2/": denied: access forbidden +``` + +To avoid this error, ensure the `Authorization` header is not stripped from the request. +For example, a proxy in front of GitLab might be redirecting to the `/jwt/auth` endpoint. diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 938093f2a27..1416dcde14f 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -33,7 +33,7 @@ Prerequisites: - You must [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. If authenticating with a personal access token or project access token, it must be - configured with the `api` scope. + configured with the `api` scope. Project access tokens must have at least the Developer role. - You must call this API endpoint serially when attempting to upload multiple files under the same package name and version. Attempts to concurrently upload multiple files into a new package name and version may face partial failures with @@ -142,7 +142,9 @@ If multiple packages have the same name, version, and filename, then the most re Prerequisites: -- You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a deploy token, it must be configured with the `read_package_registry` and/or `write_package_registry` scope. +- You need to [authenticate with the API](../../../api/rest/index.md#authentication). + - If authenticating with a deploy token, it must be configured with the `read_package_registry` and/or `write_package_registry` scope. + - Project access tokens require the `read_api` scope and at least the `Reporter` role. ```plaintext GET /projects/:id/packages/generic/:package_name/:package_version/:file_name diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 6765aa2cbb1..c8730c42022 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -24,7 +24,7 @@ Supported clients: ### Authenticate to the Package Registry -You need an token to publish a package. There are different tokens available depending on what you're trying to achieve. For more information, review the [guidance on tokens](../package_registry/index.md#authenticate-with-the-registry). +You need a token to publish a package. There are different tokens available depending on what you're trying to achieve. For more information, review the [guidance on tokens](../package_registry/index.md#authenticate-with-the-registry). Create a token and save it to use later in the process. @@ -32,6 +32,10 @@ Do not use authentication methods other than the methods documented here. Undocu #### Edit the client configuration +Update your configuration to authenticate to the Maven repository with HTTP. + +##### Custom HTTP header + You must add the authentication details to the configuration file for your client. @@ -127,6 +131,97 @@ file: } ``` +::EndTabs + +##### Basic HTTP Authentication + +You can also use basic HTTP authentication to authenticate to the Maven Package Registry. + +::Tabs + +:::TabTitle `mvn` + +| Token type | Name must be | Token | +| --------------------- | ---------------------------- | ---------------------------------------------------------------------- | +| Personal access token | The username of the user | Paste token as-is, or define an environment variable to hold the token | +| Deploy token | The username of deploy token | Paste token as-is, or define an environment variable to hold the token | +| CI Job token | `gitlab-ci-token` | `${CI_JOB_TOKEN}` | + +Add the following section to your +[`settings.xml`](https://maven.apache.org/settings.html) file. + +```xml +<settings> + <servers> + <server> + <id>gitlab-maven</id> + <username>REPLACE_WITH_NAME</username> + <password>REPLACE_WITH_TOKEN</password> + <configuration> + <authenticationInfo> + <userName>REPLACE_WITH_NAME</userName> + <password>REPLACE_WITH_TOKEN</password> + </authenticationInfo> + </configuration> + </server> + </servers> +</settings> +``` + +:::TabTitle `gradle` + +| Token type | Name must be | Token | +| --------------------- | ---------------------------- | ---------------------------------------------------------------------- | +| Personal access token | The username of the user | Paste token as-is, or define an environment variable to hold the token | +| Deploy token | The username of deploy token | Paste token as-is, or define an environment variable to hold the token | +| CI Job token | `gitlab-ci-token` | `System.getenv("CI_JOB_TOKEN")` | + +In [your `GRADLE_USER_HOME` directory](https://docs.gradle.org/current/userguide/directory_layout.html#dir:gradle_user_home), +create a file `gradle.properties` with the following content: + +```properties +gitLabPrivateToken=REPLACE_WITH_YOUR_TOKEN +``` + +Add a `repositories` section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html). + +- In Groovy DSL: + + ```groovy + repositories { + maven { + url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(PasswordCredentials) { + username = 'REPLACE_WITH_NAME' + password = gitLabPrivateToken + } + authentication { + basic(BasicAuthentication) + } + } + } + ``` + +- In Kotlin DSL: + + ```kotlin + repositories { + maven { + url = uri("https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven") + name = "GitLab" + credentials(BasicAuthentication::class) { + username = "REPLACE_WITH_NAME" + password = findProperty("gitLabPrivateToken") as String? + } + authentication { + create("basic", BasicAuthentication::class) + } + } + } + ``` + :::TabTitle `sbt` | Token type | Name must be | Token | diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 9d789c27d1f..43defb29fd5 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -87,6 +87,10 @@ Your package should now publish to the Package Registry. When publishing by using a CI/CD pipeline, you can use the [predefined variables](../../../ci/variables/predefined_variables.md) `${CI_PROJECT_ID}` and `${CI_JOB_TOKEN}` to authenticate with your project's Package Registry. We use these variables to create a `.npmrc` file [for authentication](#authenticating-via-the-npmrc) during execution of your CI/CD job. +WARNING: +When generating the `.npmrc` file, do not specify the port after `${CI_SERVER_HOST}` if it is a default port, +such as `80` for a URL starting with `http` or `443` for a URL starting with `https`. + In the GitLab project containing your `package.json`, edit or create a `.gitlab-ci.yml` file. For example: ```yaml @@ -98,8 +102,8 @@ stages: publish-npm: stage: deploy script: - - echo "@scope:registry=https://${CI_SERVER_HOST}:${CI_SERVER_PORT}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/" > .npmrc - - echo "//${CI_SERVER_HOST}:${CI_SERVER_PORT}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}" >> .npmrc + - echo "@scope:registry=https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/" > .npmrc + - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}" >> .npmrc - npm publish ``` @@ -265,7 +269,7 @@ npm deprecate @scope/package "" ### Package forwarding to npmjs.com -When an npm package is not found in the Package Registry, the request is forwarded to [npmjs.com](https://www.npmjs.com/). +When an npm package is not found in the Package Registry, the request is forwarded to [npmjs.com](https://www.npmjs.com/). The forward is performed by sending an HTTP redirect back to the requesting client. Administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md). diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index f5430c5328c..8db79dc6c5f 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -434,14 +434,19 @@ the existing package is overwritten. ### Do not allow duplicate NuGet packages -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/293748) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `nuget_duplicates_option`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/293748) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `nuget_duplicates_option`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/419078) in GitLab 16.6. Feature flag `nuget_duplicates_option` removed. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `nuget_duplicates_option`. -The feature is not ready for production use. +To prevent users from publishing duplicate NuGet packages, you can use the [GraphQl API](../../../api/graphql/reference/index.md#packagesettings) or the UI. -To prevent users from publishing duplicate NuGet packages, you can use the [GraphQl API](../../../api/graphql/reference/index.md#packagesettings). +In the UI: + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Packages and registries**. +1. In the **NuGet** row of the **Duplicate packages** table, turn off the **Allow duplicates** toggle. +1. Optional. In the **Exceptions** text box, enter a regular expression that matches the names and versions of packages to allow. + +Your changes are automatically saved. WARNING: If the .nuspec file isn't located in the root of the package, the package might diff --git a/doc/user/packages/package_registry/supported_functionality.md b/doc/user/packages/package_registry/supported_functionality.md index 3e8852da808..eb6b415ee06 100644 --- a/doc/user/packages/package_registry/supported_functionality.md +++ b/doc/user/packages/package_registry/supported_functionality.md @@ -160,9 +160,9 @@ The following authentication protocols are supported: | Package type | Supported auth protocols | |-------------------------------------------------------|-------------------------------------------------------------| -| [Maven (with `mvn`)](../maven_repository/index.md) | Headers, Basic auth ([pulling](#pulling-packages) only) (1) | -| [Maven (with `gradle`)](../maven_repository/index.md) | Headers, Basic auth ([pulling](#pulling-packages) only) (1) | -| [Maven (with `sbt`)](../maven_repository/index.md) | Basic auth (1) | +| [Maven (with `mvn`)](../maven_repository/index.md) | Headers, Basic auth | +| [Maven (with `gradle`)](../maven_repository/index.md) | Headers, Basic auth | +| [Maven (with `sbt`)](../maven_repository/index.md) | Basic auth ([pulling](#pulling-packages) only) (1) | | [npm](../npm_registry/index.md) | OAuth | | [NuGet](../nuget_repository/index.md) | Basic auth | | [PyPI](../pypi_repository/index.md) | Basic auth | diff --git a/doc/user/permissions.md b/doc/user/permissions.md index a83ce6a56c6..ab26e490f51 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -195,16 +195,16 @@ The following table lists project permissions available for each role: | [Repository](project/repository/index.md):<br>Turn on or off protected branch push for developers | | | | ✓ | ✓ | | [Repository](project/repository/index.md):<br>Remove fork relationship | | | | | ✓ | | [Repository](project/repository/index.md):<br>Force push to protected branches (3) | | | | | | -| [Repository](project/repository/index.md):<br>Remove protected branches (3) | | | | | | +| [Repository](project/repository/index.md):<br>Remove protected branches by using the UI or API | | | | ✓ | ✓ | | [Requirements Management](project/requirements/index.md):<br>Archive / reopen | | ✓ | ✓ | ✓ | ✓ | | [Requirements Management](project/requirements/index.md):<br>Create / edit | | ✓ | ✓ | ✓ | ✓ | | [Requirements Management](project/requirements/index.md):<br>Import / export | | ✓ | ✓ | ✓ | ✓ | | [Security dashboard](application_security/security_dashboard/index.md):<br>Create issue from vulnerability finding | | | ✓ | ✓ | ✓ | | [Security dashboard](application_security/security_dashboard/index.md):<br>Create vulnerability from vulnerability finding | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability finding | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Resolve vulnerability | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Revert vulnerability to detected state | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability | | | ✓ (24) | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability finding | | | ✓ | ✓ (24) | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Resolve vulnerability | | | ✓ (24) | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Revert vulnerability to detected state | | | ✓ (24) | ✓ | ✓ | | [Security dashboard](application_security/security_dashboard/index.md):<br>Use security dashboard | | | ✓ | ✓ | ✓ | | [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability | | | ✓ | ✓ | ✓ | | [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability findings in [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | @@ -249,6 +249,7 @@ The following table lists project permissions available for each role: 21. Authors of tasks can delete them even if they don't have the Owner role, but they have to have at least the Guest role for the project. 22. You must have permission to [view the epic](group/epics/manage_epics.md#who-can-view-an-epic). 23. In GitLab 15.9 and later, users with the Guest role and an Ultimate license can view private repository content if an administrator (on self-managed) or group owner (on GitLab.com) gives those users permission. The administrator or group owner can create a [custom role](custom_roles.md) through the API and assign that role to the users. +24. In GitLab 16.4 the ability for `Developers` to change the status of a vulnerability (`admin_vulnerability`) was [deprecated](../update/deprecations.md#deprecate-change-vulnerability-status-from-the-developer-role). The `admin_vulnerability` permission will be removed, by default, from all `Developer` roles in GitLab 17.0. <!-- markdownlint-enable MD029 --> diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md index ca55ab758da..94217f985cf 100644 --- a/doc/user/product_analytics/index.md +++ b/doc/user/product_analytics/index.md @@ -32,7 +32,7 @@ Product analytics uses several tools: - [**Snowplow**](https://docs.snowplow.io/docs) - A developer-first engine for collecting behavioral data, and passing it through to ClickHouse. - [**ClickHouse**](https://clickhouse.com/docs) - A database suited to store, query, and retrieve analytical data. -- [**Cube**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in Clickhouse. +- [**Cube**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in ClickHouse. The following diagram illustrates the product analytics flow: @@ -46,7 +46,7 @@ flowchart TB B --Pass data through--> C[Snowplow Enricher] end subgraph Data warehouse - C --Transform and enrich data--> D([Clickhouse]) + C --Transform and enrich data--> D([ClickHouse]) end subgraph Data visualization with dashboards E([Dashboards]) --Generated from the YAML definition--> F[Panels/Visualizations] @@ -101,11 +101,35 @@ Prerequisites: 1. Expand **Configure** and enter the configuration values. 1. Select **Save changes**. -## Instrument a GitLab project +## Onboard a GitLab project + +Onboarding a GitLab project means preparing it to receive events that are used for product analytics. + +To onboard a project: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Analyze > Analytics dashboards**. +1. Under **Product analytics**, select **Set up**. +1. Select **Set up product analytics**. +Your instance is being created, and the project onboarded. + +### Onboard an internal project + +GitLab team members can enable Product Analytics on their internal projects on GitLab.com (Ultimate) during the experiment phase. + +1. Send a message to the Product Analytics team (`#g_analyze_product_analytics`) informing them of the repository to be enabled. +1. Using ChatOps, enable both the `product_analytics_dashboards` and `combined_analytics_dashboards`: + + ```plaintext + /chatops run feature set product_analytics_dashboards true --project=FULLPATH_TO_PROJECT + /chatops run feature set combined_analytics_dashboards true --project=FULLPATH_TO_PROJECT + ``` + +## Instrument your application To instrument code to collect data, use one or more of the existing SDKs: -- [Browser SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-browser) +- [Browser SDK](instrumentation/browser_sdk.md) - [Ruby SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-rb) - [Python SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-python) - [Node SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-node) @@ -273,18 +297,24 @@ POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi If the request is successful, the returned JSON includes an array of rows of results. -## Onboarding GitLab internal projects +## View product analytics usage quota -GitLab team members can enable Product Analytics on their own internal projects on GitLab.com during the experiment phase. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/424153) in GitLab 16.6 with a [flag](../../administration/feature_flags.md) named `product_analytics_usage_quota`. Disabled by default. -1. Send a message to the Product Analytics team (`#g_analyze_product_analytics`) informing them of the repository to be enabled. -1. Ensure that the project is within an Ultimate namespace. -1. Using ChatOps, enable both the `product_analytics_dashboards` and `combined_analytics_dashboards` +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_usage_quota`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. - ```plaintext - /chatops run feature set product_analytics_dashboards true --project=FULLPATH_TO_PROJECT - /chatops run feature set combined_analytics_dashboards true --project=FULLPATH_TO_PROJECT - ``` +Product analytics usage quota is calculated from the number of events received from instrumented applications. +The tab displays the monthly totals for the group, and a breakdown of usage per project. Current month shows events counted to date. + +To view product analytics usage quota: + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Usage quota** and select the **Product analytics** tab. + +The usage quota excludes projects that are not onboarded with product analytics. ## Troubleshooting diff --git a/doc/user/product_analytics/instrumentation/browser_sdk.md b/doc/user/product_analytics/instrumentation/browser_sdk.md new file mode 100644 index 00000000000..f2beafab8e0 --- /dev/null +++ b/doc/user/product_analytics/instrumentation/browser_sdk.md @@ -0,0 +1,282 @@ +--- +stage: Analyze +group: Analytics Instrumentation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Browser SDK + +This SDK is for instrumenting web sites and applications to send data for the GitLab [product analytics functionality](../index.md). + +## How to use the Browser SDK + +### Using the NPM package + +Add the NPM package to your package JSON using your preferred package manager: + +::Tabs + +:::TabTitle yarn + +```shell +yarn add @gitlab/application-sdk-browser +``` + +:::TabTitle npm + +```shell +npm i @gitlab/application-sdk-browser +``` + +::EndTabs + +Then, for browser usage import the client SDK: + +```javascript +import { glClientSDK } from '@gitlab/application-sdk-browser'; + +this.glClient = glClientSDK({ appId, host }); +``` + +### Using the script directly + +Add the script to the page and assign the client SDK to `window`: + +```html +<script src="https://unpkg.com/@gitlab/application-sdk-browser/dist/gl-sdk.min.js"></script> +<script> + window.glClient = window.glSDK.glClientSDK({ + appId: 'YOUR_APP_ID', + host: 'YOUR_HOST', + }); +</script> +``` + +You can use a specific version of the SDK like this: + +```html +<script src="https://unpkg.com/@gitlab/application-sdk-browser@0.2.5/dist/gl-sdk.min.js"></script> +``` + +## Browser SDK initialization options + +Apart from `appId` and `host`, you can configure the Browser SDK with the following options: + +```typescript +interface GitLabClientSDKOptions { + appId: string; + host: string; + hasCookieConsent?: boolean; + respectGlobalPrivacyControl?: boolean; + trackerId?: string; + pagePingTracking?: + | boolean + | { + minimumVisitLength?: number; + heartbeatDelay?: number; + }; + plugins?: AllowedPlugins; +} +``` + +| Option | Description | +| :---------------------------- | :---------- | +| `appId` | The ID provided by the GitLab Project Analytics setup guide. This ID ensures your data is sent to your analytics instance. | +| `host` | The GitLab Project Analytics instance provided by the setup guide. | +| `hasCookieConsent` | Whether to use cookies to identify unique users and record their full IP address. Set to `false` by default. When `false`, users are considered anonymous users. No cookies or other storage mechanisms are used to identify users. | +| `respectGlobalPrivacyControl` | Whether to respect the user's [GPC](https://globalprivacycontrol.org/) configuration to permit or refuse tracking. Set to `true` by default. When `false`, events are emitted regardless of user configuration. | +| `trackerId` | Used to differentiate between multiple trackers running on the same page or application, because each tracker instance can be configured differently to capture different sets of data. This identifier helps ensure that the data sent to the collector is correctly associated with the correct tracker configuration. Default value is `gitlab`. | +| `pagePingTracking` | Option to track user engagement on your website or application by sending periodic events while a user is actively browsing a page. Page pings provide valuable insight into how users interact with your content, such as how long they spend on a page, which sections they are viewing, and whether they are scrolling. `pagePingTracking` can be boolean or an object. As a boolean, set to `true` it enables page ping with default options, and set to `false` it disables page ping tracking. As an object, it has two options: `minimumVisitLength` (the minimum time that must have elapsed before the first heartbeat) and `heartbeatDelay` (the interval at which the callback is fired). | +| `plugins` | Specify which plugins to enable or disable. By default all plugins are enabled. | + +### Plugins + +- `Client Hints`: An alternative to tracking the User Agent, which is particularly useful in browsers that are freezing the User Agent string. +Enabling this plugin will automatically capture the following context: + + For example, + [iglu:org.ietf/http_client_hints/jsonschema/1-0-0](https://github.com/snowplow/iglu-central/blob/master/schemas/org.ietf/http_client_hints/jsonschema/1-0-0) + has the following configuration: + + ```json + { + "isMobile":false, + "brands":[ + { + "brand":"Google Chrome", + "version":"89" + }, + { + "brand":"Chromium", + "version":"89" + } + ] + } + ``` + +- `Link Click Tracking`: With this plugin, the tracker adds click event listeners to all link elements. Link clicks are tracked as self-describing events. Each link-click event captures the link's `href` attribute. The event also has fields for the link's ID, classes, and target (where the linked document is opened, such as a new tab or new window). + +- `Performance Timing`: It collects performance-related data from a user's browser using the `Navigation Timing API`. This API provides detailed information about the various stages of loading a web page, such as domain lookup, connection time, content download, and rendering times. This plugin helps to gather insights into how well a website performs for users, identify potential performance bottlenecks, and improve the overall user experience. + +- `Error Tracking`: It helps to capture and track errors that occur on a website or application. By monitoring these errors, you can gain insights into potential issues with code or third-party libraries, which can help to improve the overall user experience, and maintain the quality of the website or application. + +By default all plugins are enabled. You can disable or enable these plugins through the `plugins` object: + +```typescript +const tracker = glClientSDK({ + ...options, + plugins: { + clientHints: true, + linkTracking: true, + performanceTiming: true, + errorTracking: true, + }, +}); +``` + +## Methods + +### `identify` + +Used to associate a user and their attributes with the session and tracking events. + +```javascript +glClient.identify(userId, userAttributes); +``` + +| Property | Type | Description | +| :--------------- | :-------------------------- | :---------------------------------------------------------------------------- | +| `userId` | `String` | The user identifier your application uses to identify individual users. | +| `userAttributes` | `Object`/`Null`/`undefined` | The user attributes that need to be added to the session and tracking events. | + +### `page` + +Used to trigger a pageview event. + +```javascript +glClient.page(eventAttributes); +``` + +| Property | Type | Description | +| :---------------- | :-------------------------- | :---------------------------------------------------------------- | +| `eventAttributes` | `Object`/`Null`/`undefined` | The event attributes that need to be added to the pageview event. | + +The `eventAttributes` object supports the following optional properties: + +| Property | Type | Description | +| :--------------- | :-------------------------- | :---------------------------------------------------------------------------- | +| `title` | `String` | Override the default page title. | +| `contextCallback` | `Function` | A callback that fires on the page view. | +| `context` | `Object` | Add context (additional information) on the page view. | +| `timestamp` | `timestamp` | Set the true timestamp or overwrite the device-sent timestamp on an event. | + +### `track` + +Used to trigger a custom event. + +```javascript +glClient.track(eventName, eventAttributes); +``` + +| Property | Type | Description | +| :---------------- | :-------------------------- | :--------------------------------------------------------------- | +| `eventName` | `String` | The name of the custom event. | +| `eventAttributes` | `Object`/`Null`/`undefined` | The event attributes that need to be added to the tracked event. | + +### `refreshLinkClickTracking` + +`enableLinkClickTracking` tracks only clicks on links that exist when the page has loaded. To track new links added to the page after it has been loaded, use `refreshLinkClickTracking`. + +```javascript +glClient.refreshLinkClickTracking(); +``` + +### `trackError` + +NOTE: +`trackError` is supported on the Browser SDK, but the resulting events are not used or available. + +Used to capture errors. This works only when the `errorTracking` plugin is enabled. The [plugin](#plugins) is enabled by default. + +```javascript +glClient.trackError(eventAttributes); +``` + +For example, `trackError` can be used in `try...catch` like below: + +```javascript +try { + // Call the function that throws an error + throwError(); +} catch (error) { + glClient.trackError({ + message: error.message, // "This is a custom error" + filename: error.fileName || 'unknown', // The file in which the error occurred (e.g., "index.html") + lineno: error.lineNumber || 0, // The line number where the error occurred (e.g., 2) + colno: error.columnNumber || 0, // The column number where the error occurred (e.g., 6) + error: error, // The Error object itself + }); +} +``` + +| Property | Type | Description | +| :---------------- | :------- | :------------------------------------------------------------------------------------------------------------------- | +| `eventAttributes` | `Object` | The event attributes that need to be added to the tracked event. `message` is a mandatory key in `eventAttributes`. | + +### `addCookieConsent` + +`addCookieConsent` is used to allow tracking of user identifiers via cookies. By default `hasCookieConsent` is false, and no user identifiers are passed. To enable tracking of user identifiers, call the `addCookieConsent` method. This step is not needed if you intialized the Browser SDK with `hasCookieConsent` set to true. + +```javascript +glClient.addCookieConsent(); +``` + +### `setCustomUrl` + +Used to set a custom URL for tracking. + +```javascript +glClient.setCustomUrl(url); +``` + +| Property | Type | Description | +| :------- | :------- | :------------------------------------------------ | +| `url` | `String` | The custom URL that you want to set for tracking. | + +### `setReferrerUrl` + +Used to set a referrer URL for tracking. + +```javascript +glClient.setReferrerUrl(url); +``` + +| Property | Type | Description | +| :------- | :------- | :-------------------------------------------------- | +| `url` | `String` | The referrer URL that you want to set for tracking. | + +### `setDocumentTitle` + +Used to override the document title. + +```javascript +glClient.setDocumentTitle(title); +``` + +| Property | Type | Description | +| :------- | :------- | :--------------------------------- | +| `title` | `String` | The document title you want to set. | + +## Contribute + +If you would like to contribute to Browser SDK, follow the [contributing guide](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-js/-/blob/main/docs/Contributing.md). + +## Troubleshooting + +If the Browser SDK is not sending events, or behaving in an unexpected way, take the following actions: + +1. Verify that the `appId` and host values in the options object are correct. +1. Check if any browser privacy settings, extensions, or ad blockers are interfering with the Browser SDK. + +For more information and assistance, see the [Snowplow documentation](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/browser-tracker/browser-tracker-v3-reference/) +or contact the [Analytics Instrumentation team](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/#team-members). diff --git a/doc/user/product_analytics/instrumentation/index.md b/doc/user/product_analytics/instrumentation/index.md new file mode 100644 index 00000000000..f909a01ff59 --- /dev/null +++ b/doc/user/product_analytics/instrumentation/index.md @@ -0,0 +1,15 @@ +--- +stage: Analyze +group: Analytics Instrumentation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Instrumentation + +To instrument an application to send events to GitLab product analytics you can use one of the following language and platform specific tracking SDKs: + +- [Browser SDK](browser_sdk.md) +- [Ruby SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-rb) +- [Python SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-python) +- [Node SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-node) +- [.NET SDK](https://gitlab.com/gitlab-org/analytics-section/product-analytics/gl-application-sdk-dotnet) diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index d41eee911f9..70c12cbcf00 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -54,10 +54,9 @@ Using the **Delete user and contributions** option may result in removing more d When deleting users, you can either: -- Delete just the user. Not all associated records are deleted with the user. Instead of being deleted, these records - are moved to a system-wide user with the username Ghost User. The Ghost User's purpose is to act as a container for - such records. Any commits made by a deleted user still display the username of the original user. - The user's personal projects are deleted, not moved to the Ghost User. +- Delete just the user, but move contributions to a system-wide "Ghost User": + - The `@ghost` acts as a container for all deleted users' contributions. + - The user's profile and personal projects are deleted, instead of moved to the Ghost User. - Delete the user and their contributions, including: - Abuse reports. - Emoji reactions. @@ -74,6 +73,9 @@ When deleting users, you can either: [merge requests](../../project/merge_requests/index.md) and [snippets](../../snippets.md). +In both cases, commits retain [user information](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects#_git_commit_objects) +and therefore data integrity within a [Git repository](../../project/repository/index.md). + An alternative to deleting is [blocking a user](../../../administration/moderate_users.md#block-a-user). When a user is deleted from an [abuse report](../../../administration/review_abuse_reports.md) or spam log, these associated diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index d1f1d28663e..d26f2193124 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -544,3 +544,9 @@ generates the codes. For example: 1. Select General. 1. Select Date & Time. 1. Enable Set Automatically. If it's already enabled, disable it, wait a few seconds, and re-enable. + +### Error: "Permission denied (publickey)" when regenerating recovery codes + +If you receive a `Permission denied (publickey)` error when attempting to [generate new recovery codes using an SSH key](#generate-new-recovery-codes-using-ssh) +and you are using a non-default SSH key pair file path, +you might need to [manually register your private SSH key](../../ssh.md#configure-ssh-to-point-to-a-different-directory) using `ssh-agent`. diff --git a/doc/user/profile/comment_templates.md b/doc/user/profile/comment_templates.md index 50df5f8fdb4..98fabdb0a35 100644 --- a/doc/user/profile/comment_templates.md +++ b/doc/user/profile/comment_templates.md @@ -10,10 +10,7 @@ type: howto > - GraphQL support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352956) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. > - User interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113232) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. Enabled for GitLab team members only. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119468) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `saved_replies`. -On GitLab.com, this feature is available. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123363) in GitLab 16.6. With comment templates, create and reuse text for any text area in: @@ -25,7 +22,7 @@ With comment templates, create and reuse text for any text area in: Comment templates can be small, like approving a merge request and unassigning yourself from it, or large, like chunks of boilerplate text you use frequently: - + ## Use comment templates in a text area @@ -65,4 +62,4 @@ To edit or delete a previously comment template: 1. On the left sidebar, select **Comment templates** (**{comment-lines}**). 1. Scroll to **My comment templates**, and identify the comment template you want to edit. 1. To edit, select **Edit** (**{pencil}**). -1. To delete, select **Delete** (**{remove}**), then select **Delete** again from the modal window. +1. To delete, select **Delete** (**{remove}**), then select **Delete** again on the dialog. diff --git a/doc/user/profile/img/comment_template_v16_6.png b/doc/user/profile/img/comment_template_v16_6.png Binary files differnew file mode 100644 index 00000000000..7990ca604ce --- /dev/null +++ b/doc/user/profile/img/comment_template_v16_6.png diff --git a/doc/user/profile/img/saved_replies_dropdown_v16_0.png b/doc/user/profile/img/saved_replies_dropdown_v16_0.png Binary files differdeleted file mode 100644 index 4608484a496..00000000000 --- a/doc/user/profile/img/saved_replies_dropdown_v16_0.png +++ /dev/null diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 6536a992292..64fa5d7b448 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -62,6 +62,10 @@ To add new email to your account: 1. Select **Add email address**. 1. Verify your email address with the verification email received. +NOTE: +[Making your email non-public](#set-your-public-email) does not prevent it from being used for commit matching, +[project imports](../project/import/index.md), and [group migrations](../group/import/index.md). + ## Make your user profile page private You can make your user profile visible to only you and GitLab administrators. @@ -128,6 +132,8 @@ to match your username. ## Add external accounts to your user profile page +> Mastodon user account [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/132892) in 16.6 [with a flag](../feature_flags.md) named `mastodon_social_ui`. Disabled by default. This feature is in [Beta](../../policy/experiment-beta-support.md#beta). + You can add links to certain other external accounts you might have, like Skype and Twitter. They can help other users connect with you on other platforms. @@ -138,6 +144,7 @@ To add links to other accounts: 1. In the **Main settings** section, add your: - Discord [user ID](https://support.discord.com/hc/en-us/articles/206346498-Where-can-I-find-my-User-Server-Message-ID-). - LinkedIn profile name. + - Mastodon username. - Skype username. - Twitter @username. diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index 706065d4693..8d34055d42c 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -9,6 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Enhanced email styling [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78604) in GitLab 14.9 [with a feature flag](../../administration/feature_flags.md) named `enhanced_notify_css`. Disabled by default. > - Enhanced email styling [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/355907) in GitLab 14.9. > - Enhanced email styling [enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/355907) in GitLab 15.0. +> - Product marketing emails [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/418137) in GitLab 16.6. Stay informed about what's happening in GitLab with email notifications. You can receive updates about activity in issues, merge requests, epics, and designs. @@ -84,8 +85,6 @@ different values for a project or a group. - **Notification email**: the email address your notifications are sent to. Defaults to your primary email address. -- **Receive product marketing emails**: select this checkbox to receive - [periodic emails](#opt-out-of-product-marketing-emails) about GitLab features. - **Global notification level**: the default [notification level](#notification-levels) which applies to all your notifications. - **Receive notifications about your own activity**: select this checkbox to receive @@ -145,32 +144,6 @@ Or: <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> To learn how to be notified when a new release is available, watch [Notification for releases](https://www.youtube.com/watch?v=qyeNkGgqmH4). -### Opt out of product marketing emails - -You can receive emails that teach you about various GitLab features. -These emails are enabled by default. - -To opt out: - -1. On the left sidebar, select your avatar. -1. Select **Preferences**. -1. On the left sidebar, select **Notifications**. -1. Clear the **Receive product marketing emails** checkbox. - Edited settings are automatically saved and enabled. - -Disabling these emails does not disable all emails. -Learn how to [opt out of all emails from GitLab](#opt-out-of-all-gitlab-emails). - -#### Self-managed product marketing emails **(FREE SELF)** - -The self-managed installation generates and automatically sends these emails based on user actions. -Turning this on does not cause your GitLab instance or your company to send any personal information to -GitLab Inc. - -An instance administrator can configure this setting for all users. If you choose to opt out, your -setting overrides the instance-wide setting, even when an administrator later enables these emails -for all users. - ## Notification events Users are notified of the following events: @@ -348,7 +321,6 @@ If you no longer wish to receive any email notifications: 1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. -1. Clear the **Receive product marketing emails** checkbox. 1. Set your **Global notification level** to **Disabled**. 1. Clear the **Receive notifications about your own activity** checkbox. 1. If you belong to any groups or projects, set their notification setting to **Global** or diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 9135a142612..a953a878cc9 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -137,6 +137,42 @@ Personal access tokens expire on the date you define, at midnight, 00:00 AM UTC. - In GitLab Ultimate, administrators can [limit the allowable lifetime of access tokens](../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens). If not set, the maximum allowable lifetime of a personal access token is 365 days. - In GitLab Free and Premium, the maximum allowable lifetime of a personal access token is 365 days. +- If you do not set an expiry date when creating a personal access token, the expiry date is set to the + [maximum allowed lifetime for the token](../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens). + If the maximum allowed lifetime is not set, the default expiry date is 365 days from the date of creation. + +### Service Accounts + +You can [create a personal access token for a service account](../../api/groups.md#create-personal-access-token-for-service-account-user) with no expiry date. + +NOTE: +Allowing personal access tokens for service accounts to be created with no expiry date only affects tokens created after you change this setting. It does not affect existing tokens. + +#### GitLab.com + +Prerequisite: + +- You must have the Owner role in the top-level group. + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Permissions and group features**. +1. Clear the **Service account token expiration** checkbox. + +You can now create personal access tokens for a service account user with no expiry date. + +#### Self-managed GitLab + +Prerequisite: + +- You must be an administrator for your self-managed instance. + +1. On the left sidebar, select **Search or go to**. +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Clear the **Service account token expiration** checkbox. + +You can now create personal access tokens for a service account user with no expiry date. ## Create a personal access token programmatically **(FREE SELF)** diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 170545d851f..34f083e0b48 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -268,6 +268,22 @@ To use exact times on the GitLab UI: 1. Clear the **Use relative times** checkbox. 1. Select **Save changes**. +### Customize time format + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15206) in GitLab 16.6. + +You can customize the format used to display times of activities on your group and project overview pages and user profiles. You can display times as: + +- 12 hour format. For example: `2:34 PM`. +- 24 hour format. For example: `14:34`. + +To customize the time format: + +1. On the left sidebar, select your avatar. +1. Select **Preferences** > **Time preferences**. +1. In **Time format**, select either the **12-hour** or **24-hour** option. +1. Select **Save changes**. + ## User identities in CI job JSON web tokens > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387537) in GitLab 16.0. diff --git a/doc/user/profile/service_accounts.md b/doc/user/profile/service_accounts.md index 6bb96b9c552..8fa0067f150 100644 --- a/doc/user/profile/service_accounts.md +++ b/doc/user/profile/service_accounts.md @@ -53,6 +53,8 @@ Prerequisite: You define the scopes for the service account by [setting the scopes for the personal access token](personal_access_tokens.md#personal-access-token-scopes). + Optional. You can [create a personal access token with no expiry date](personal_access_tokens.md#when-personal-access-tokens-expire). + The response includes the personal access token value. 1. Make this service account a group or project member by [manually adding the service account user to the group or project](#add-a-service-account-to-subgroup-or-project). @@ -74,6 +76,8 @@ Prerequisite: You define the scopes for the service account by [setting the scopes for the personal access token](personal_access_tokens.md#personal-access-token-scopes). + Optional. You can [create a personal access token with no expiry date](personal_access_tokens.md#when-personal-access-tokens-expire). + The response includes the personal access token value. 1. Make this service account a group or project member by diff --git a/doc/user/project/codeowners/index.md b/doc/user/project/codeowners/index.md index d783471f0da..0fa9983e93b 100644 --- a/doc/user/project/codeowners/index.md +++ b/doc/user/project/codeowners/index.md @@ -54,6 +54,10 @@ GitLab shows the Code Owners at the top of the page. ## Set up Code Owners +Prerequisites: + +- You must be able to either push to the default branch or create a merge request. + 1. Create a `CODEOWNERS` file in your [preferred location](#codeowners-file). 1. Define some rules in the file following the [Code Owners syntax reference](reference.md). Some suggestions: @@ -145,7 +149,7 @@ of the merge request becomes optional. Inviting **Subgroup Y** to a parent group of **Project A** [is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/288851). To set **Subgroup Y** as -Code Owners [invite this group directly to the project](#inviting-subgroups-to-projects-in-parent-groups) itself. +Code Owners, [invite this group directly to the project](#inviting-subgroups-to-projects-in-parent-groups) itself. NOTE: For approval to be required, groups as Code Owners must have a direct membership @@ -196,7 +200,7 @@ You can organize Code Owners by putting them into named sections. You can use sections for shared directories, so that multiple teams can be reviewers. -To add a section to the `CODEOWNERS` file, enter a section name in brackets, +To add a section to the `CODEOWNERS` file, enter a section name in square brackets, followed by the files or directories, and users, groups, or subgroups: ```plaintext @@ -206,7 +210,7 @@ internal/README.md @user2 ``` Each Code Owner in the merge request widget is listed under a label. -The following image shows a **Groups** and **Documentation** section: +The following image shows **Groups** and **Documentation** sections:  @@ -221,7 +225,9 @@ All paths in that section inherit this default, unless you override the section default on a specific line. Default owners are applied when specific owners are not specified for file paths. -Specific owners defined beside the file path override default owners: +Specific owners defined beside the file path override default owners. + +For example: ```plaintext [Documentation] @docs-team @@ -259,8 +265,8 @@ config/db/database-setup.md @docs-team #### Use regular entries and sections together -If you set a default Code Owner for a path outside a section, their approval is always required, and -the entry isn't overridden. +If you set a default Code Owner for a path **outside a section**, their approval is always required. +Such entries aren't overridden by sections. Entries without sections are treated as if they were another, unnamed section: ```plaintext @@ -287,7 +293,7 @@ In this example: of the `@general-approvers`,`@docs-team`, and `@database-team` groups. Compare this behavior to when you use only [default owners for sections](#set-default-owner-for-a-section), -when specific entries within a section override the section default. +when specific entries in a section override the section default. #### Sections with duplicate names @@ -313,13 +319,14 @@ entries under **Database**. The entries defined under the sections **Documentati #### Make a Code Owners section optional -You can designate optional sections in your Code Owners file. Prepend the -section name with the caret `^` character to treat the entire section as optional. +You can designate optional sections in your Code Owners file. Optional sections enable you to designate responsible parties for various parts of your codebase, but not require approval from them. This approach provides a more relaxed policy for parts of your project that are frequently updated, but don't require stringent reviews. +To treat the entire section as optional, prepend the section name with the caret `^` character. + In this example, the `[Go]` section is optional: ```plaintext @@ -333,7 +340,7 @@ In this example, the `[Go]` section is optional: *.go @root ``` -The optional Code Owners section displays in merge requests under the **Approval Rules** area: +The optional Code Owners section displays in merge requests under the description:  @@ -348,18 +355,25 @@ section is marked as optional. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335451) in GitLab 15.9. -You can require multiple approvals for the Code Owners sections under the Approval Rules area in merge requests. -Append the section name with a number `n` in brackets. This requires `n` approvals from the Code Owners in this section. +You can require multiple approvals for the Code Owners sections in the Approvals area in merge requests. +Append the section name with a number `n` in brackets, for example, `[2]` or `[3]`. +This requires `n` approvals from the Code Owners in this section. Valid entries for `n` are integers `≥ 1`. `[1]` is optional because it is the default. Invalid values for `n` are treated as `1`. WARNING: -[Issue #384881](https://gitlab.com/gitlab-org/gitlab/-/issues/385881) proposes changes +[Issue 384881](https://gitlab.com/gitlab-org/gitlab/-/issues/385881) proposes changes to the behavior of this setting. Do not intentionally set invalid values. They may -become valid in the future, and cause unexpected behavior. +become valid in the future and cause unexpected behavior. + +To require multiple approvals from Code Owners: -Make sure you enabled `Require approval from code owners` in `Settings > Repository > Protected branches`, otherwise the Code Owner approvals are optional. +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > Repository**. +1. Expand **Protected branches**. +1. Next to the default branch, turn on the toggle under **Code owner approval**. +1. Edit the `CODEOWNERS` file to add a rule for multiple approvals. -In this example, the `[Documentation]` section requires 2 approvals: +For example, to require two approvals for the `[Documentation]` section: ```plaintext [Documentation][2] @@ -369,7 +383,7 @@ In this example, the `[Documentation]` section requires 2 approvals: *.rb @dev-team ``` -The `Documentation` Code Owners section under the **Approval Rules** area displays 2 approvals are required: +The `Documentation` Code Owners section in the Approvals area displays two approvals are required:  @@ -377,7 +391,7 @@ The `Documentation` Code Owners section under the **Approval Rules** area displa Users who are **Allowed to push** can choose to create a merge request for their changes, or push the changes directly to a branch. If the user -skips the merge request process, the protected-branch features +skips the merge request process, the protected branch features and Code Owner approvals built into merge requests are also skipped. This permission is often granted to accounts associated with diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 8b7e185508b..351762228fb 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -88,7 +88,8 @@ Create a deploy token to automate deployment tasks that can run independently of Prerequisites: -- You must have at least the Maintainer role for the project or group. +- To create a group deploy token, you must have the Owner role for the group. +- To create a project deploy token, you must have at least the Maintainer role for the project. 1. On the left sidebar, select **Search or go to** and find your project or group. 1. Select **Settings > Repository**. @@ -106,7 +107,8 @@ Revoke a token when it's no longer required. Prerequisites: -- You must have at least the Maintainer role for the project or group. +- To revoke a group deploy token, you must have the Owner role for the group. +- To revoke a project deploy token, you must have at least the Maintainer role for the project. To revoke a deploy token: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 4da756b05ea..f9b94774809 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -17,11 +17,10 @@ The namespace is a user or group in GitLab, such as `gitlab.com/sidney-jones` or `gitlab.com/customer-success`. You can use bulk actions in the rails console to move projects to different namespaces. -- If you are importing to a self-managed GitLab instance, you can use the [GitHub Rake task](../../../administration/raketasks/github_import.md) instead. The - Rake task imports projects without the constraints of a [Sidekiq](../../../development/sidekiq/index.md) worker. -- If you are importing from GitHub Enterprise to GitLab.com, use the - [GitLab Import API](../../../api/import.md#import-repository-from-github) GitHub endpoint instead. This allows you to provide a different domain to import the project from. - Using the UI, the GitHub importer always imports from the `github.com` domain. +If you are importing from GitHub Enterprise to GitLab.com, use the +[GitLab Import API](../../../api/import.md#import-repository-from-github) GitHub endpoint instead. The API allows you to +provide a different domain to import the project from. Using the UI, the GitHub importer always imports from the +`github.com` domain. When importing projects: @@ -123,9 +122,10 @@ The [GitHub integration method (above)](#use-the-github-integration) is recommen If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: -1. Go to <https://github.com/settings/tokens/new> +1. Go to `https://github.com/settings/tokens/new`. 1. Enter a token description. -1. Select the repository scope. +1. Select the `repo` scope. +1. Optional. To [import collaborators](#select-additional-items-to-import), select the `read:org` scope. 1. Select **Generate token**. 1. Copy the token hash. 1. Go back to GitLab and provide the token to the GitHub importer. diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index b2092082bf8..921669e4b70 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -23,8 +23,8 @@ GitLab imports the following information directly: Other Jira issue metadata that is not formally mapped to GitLab issue fields is imported into the GitLab issue's description as plain text. -Our parser for converting text in Jira issues to GitLab Flavored Markdown is only compatible with -Jira V3 REST API. +Text in Jira issues is not parsed to GitLab Flavored Markdown which can result in broken text formatting. +For more information, see [issue 379104](https://gitlab.com/gitlab-org/gitlab/-/issues/379104). There is an [epic](https://gitlab.com/groups/gitlab-org/-/epics/2738) tracking the addition of issue assignees, comments, and much more in the future iterations of the GitLab Jira importer. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index b60d87adbd3..9ee1e33ecdd 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -143,19 +143,24 @@ To push your repository and create a project: 1. Push with SSH or HTTPS: - To push with SSH: - ```shell - git push --set-upstream git@gitlab.example.com:namespace/myproject.git master - ``` + ```shell + # Use this version if your project uses the standard port 22 + $ git push --set-upstream git@gitlab.example.com:namespace/myproject.git main + + # Use this version if your project requires a non-standard port number + $ git push --set-upstream ssh://git@gitlab.example.com:00/namespace/myproject.git main + ``` - To push with HTTPS: - ```shell - git push --set-upstream https://gitlab.example.com/namespace/myproject.git master - ``` + ```shell + git push --set-upstream https://gitlab.example.com/namespace/myproject.git master + ``` - For `gitlab.example.com`, use the domain name of the machine that hosts your Git repository. - For `namespace`, use the name of your [namespace](../namespace/index.md). - For `myproject`, use the name of your project. + - If specifying a port, change `00` to your project's required port number. - Optional. To export existing repository tags, append the `--tags` flag to your `git push` command. 1. Optional. To configure the remote: diff --git a/doc/user/project/integrations/aws_codepipeline.md b/doc/user/project/integrations/aws_codepipeline.md index b081544199e..5404101b4f6 100644 --- a/doc/user/project/integrations/aws_codepipeline.md +++ b/doc/user/project/integrations/aws_codepipeline.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Import and Integrate +stage: none +group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 6f70305ce8b..abfd4243e07 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -74,7 +74,8 @@ You can use slash commands to run common GitLab operations. Replace `<project>` - You must authorize your Slack user on GitLab.com when you run your first slash command. - You can [create a shorter project alias](#create-a-project-alias-for-slash-commands) for slash commands. -**For [Slack slash commands](slack_slash_commands.md) on self-managed GitLab, [Mattermost slash commands](mattermost_slash_commands.md), and [ChatOps](../../../ci/chatops/index.md)**, replace `/gitlab` with the slash command trigger name configured for your integration. +**For [Slack slash commands](slack_slash_commands.md) on self-managed GitLab and [Mattermost slash commands](mattermost_slash_commands.md)**, +replace `/gitlab` with the slash command trigger name configured for your integration. The following slash commands are available: @@ -172,7 +173,11 @@ The following events are available for Slack notifications: ## Troubleshooting -### GitLab for Slack app does not appear in the list of integrations +When configuring the GitLab for Slack app on GitLab.com, you might encounter the following issues. + +For self-managed GitLab, see [GitLab for Slack app administration](../../../administration/settings/slack_app.md#troubleshooting). + +### The app does not appear in the list of integrations The GitLab for Slack app might not appear in the list of integrations. To have the GitLab for Slack app on your self-managed instance, an administrator must [enable the integration](../../../administration/settings/slack_app.md). On GitLab.com, the GitLab for Slack app is available by default. @@ -193,9 +198,10 @@ As a workaround, ensure: - If using a [project alias](#create-a-project-alias-for-slash-commands), the alias is correct. - The GitLab for Slack app is [enabled for the project](#from-project-integration-settings). -### Slash commands return `/gitlab failed with the error "dispatch_failed"` in Slack +### Slash commands return an error in Slack -Slash commands might return `/gitlab failed with the error "dispatch_failed"` in Slack. To resolve this issue, ensure an administrator has properly configured the [GitLab for Slack app settings](../../../administration/settings/slack_app.md) on your self-managed instance. +Slash commands might return `/gitlab failed with the error "dispatch_failed"` in Slack. +To resolve this issue, ensure an administrator has properly configured the [GitLab for Slack app settings](../../../administration/settings/slack_app.md) on your self-managed instance. ### Notifications are not received to a channel diff --git a/doc/user/project/issues/associate_zoom_meeting.md b/doc/user/project/issues/associate_zoom_meeting.md index bb8f0ccd186..e112c5ebd0d 100644 --- a/doc/user/project/issues/associate_zoom_meeting.md +++ b/doc/user/project/issues/associate_zoom_meeting.md @@ -30,7 +30,7 @@ a system alert notifies you of its successful addition. The issue's description is automatically edited to include the Zoom link, and a button appears right under the issue's title. - + You are only allowed to attach a single Zoom meeting to an issue. If you attempt to add a second Zoom meeting using the `/zoom` quick action, it doesn't work. You diff --git a/doc/user/project/issues/img/zoom-quickaction-button.png b/doc/user/project/issues/img/zoom-quickaction-button.png Binary files differdeleted file mode 100644 index 3be4f36f88f..00000000000 --- a/doc/user/project/issues/img/zoom-quickaction-button.png +++ /dev/null diff --git a/doc/user/project/issues/img/zoom_quickaction_button_v16_6.png b/doc/user/project/issues/img/zoom_quickaction_button_v16_6.png Binary files differnew file mode 100644 index 00000000000..cf869b59714 --- /dev/null +++ b/doc/user/project/issues/img/zoom_quickaction_button_v16_6.png diff --git a/doc/user/project/issues/issue_weight.md b/doc/user/project/issues/issue_weight.md index b1a1390d3d2..ddd08ee1de0 100644 --- a/doc/user/project/issues/issue_weight.md +++ b/doc/user/project/issues/issue_weight.md @@ -10,7 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w When you have a lot of issues, it can be hard to get an overview. With weighted issues, you can get a better idea of how much time, -value, or complexity a given issue has or costs. +value, or complexity a given issue has or costs. You can also [sort by weight](sorting_issue_lists.md#sorting-by-weight) +to see which issues need to be prioritized. ## View the issue weight diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 901a8fe9850..6df33a4fb06 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -190,6 +190,7 @@ To add a group to a project: 1. Select **Invite**. The members of the group are not displayed on the **Members** tab. +Private groups are masked from unauthorized users. The **Members** tab shows: - Members who are directly assigned to the project. diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index deefe9040fa..94dbb922c0b 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -76,6 +76,11 @@ In addition: - On the group's page, the project is listed on the **Shared projects** tab. - On the project's **Members** page, the group is listed on the **Groups** tab. +- From [GitLab 16.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134623), + the invited group's name and membership source will be masked unless: + - the group is public, or + - the current user is a member of the group, or + - the current user is a member of the project. - Each user is assigned a maximum role. - Members who have the **Project Invite** badge next to their profile on the usage quota page count towards the billable members of the shared project's top-level group. diff --git a/doc/user/project/merge_requests/ai_in_merge_requests.md b/doc/user/project/merge_requests/ai_in_merge_requests.md index c29060bf44b..2b4b28dafa2 100644 --- a/doc/user/project/merge_requests/ai_in_merge_requests.md +++ b/doc/user/project/merge_requests/ai_in_merge_requests.md @@ -14,7 +14,7 @@ Additional information on enabling these features and maturity can be found in o > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10591) in GitLab 16.3 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com. Merge requests in projects often have [templates](../description_templates.md#create-a-merge-request-template) defined that need to be filled out. This helps reviewers and other users understand the purpose and changes a merge request might propose. @@ -40,7 +40,7 @@ Provide feedback on this experimental feature in [issue 416537](https://gitlab.c > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10401) in GitLab 16.2 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com. GitLab Duo Merge request summaries are available on the merge request page in: @@ -56,7 +56,7 @@ Provide feedback on this experimental feature in [issue 408726](https://gitlab.c > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10466) in GitLab 16.0 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com. When you've completed your review of a merge request and are ready to [submit your review](reviews/index.md#submit-a-review), generate a GitLab Duo Code review summary: @@ -78,7 +78,7 @@ Provide feedback on this experimental feature in [issue 408991](https://gitlab.c > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10453) in GitLab 16.2 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com. When preparing to merge your merge request you may wish to edit the proposed squash or merge commit message. @@ -99,7 +99,7 @@ Provide feedback on this experimental feature in [issue 408994](https://gitlab.c > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10366) in GitLab 16.0 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `code-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com. Use GitLab Duo Test generation in a merge request to see a list of suggested tests for the file you are reviewing. This functionality can help determine if appropriate test coverage has been provided, or if you need more coverage for your project. diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index ae16eb2a790..3be546faabe 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -29,8 +29,8 @@ These settings limit who can approve merge requests: Prevents users who add commits to a merge request from also approving it. - [**Prevent editing approval rules in merge requests**](#prevent-editing-approval-rules-in-merge-requests): Prevents users from overriding project level approval rules on merge requests. -- [**Require user password to approve**](#require-user-password-to-approve): - Force potential approvers to first authenticate with a password. +- [**Require user re-authentication (password or SAML) to approve**](#require-user-re-authentication-to-approve): + Force potential approvers to first authenticate with either a password or with SAML. - Code Owner approval removals: Define what happens to existing approvals when commits are added to the merge request. - **Keep approvals**: Do not remove any approvals. @@ -104,20 +104,29 @@ on merge requests, you can disable this setting: This change affects all open merge requests. -## Require user password to approve +## Require user re-authentication to approve > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5981) in GitLab 12.0. > - Moved to GitLab Premium in 13.9. +> - SAML authentication for GitLab.com groups [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5981) in GitLab 16.6. -You can force potential approvers to first authenticate with a password. This +You can force potential approvers to first authenticate with either: + +- A password. +- SAML. Available on GitLab.com groups only. + +This permission enables an electronic signature for approvals, such as the one defined by [Code of Federal Regulations (CFR) Part 11](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=11&showFR=1&subpartNode=21:1.0.1.1.8.3)): -1. Enable password authentication for the web interface, as described in the - [sign-in restrictions documentation](../../../../administration/settings/sign_in_restrictions.md#password-authentication-enabled). +1. Enable password authentication and SAML authentication (available only on GitLab.com groups). For more information on: + - Password authentication, see + [sign-in restrictions documentation](../../../../administration/settings/sign_in_restrictions.md#password-authentication-enabled). + - SAML authentication for GitLab.com groups, see + [SAML SSO for GitLab.com groups documentation](../../../../user/group/saml_sso). 1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Merge request approvals** section, scroll to **Approval settings** and - select **Require user password to approve**. + select **Require user re-authentication (password or SAML) to approve**. 1. Select **Save changes**. ## Remove all approvals when commits are added to the source branch diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index ef1554f3b86..af76aa100c1 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -50,7 +50,18 @@ Commit `G` is added after the cherry-pick. ## Cherry-pick all changes from a merge request After a merge request is merged, you can cherry-pick all changes introduced -by the merge request: +by the merge request. + +Prerequisites: + +- You must have a role in the project that allows you to edit merge requests, and add + code to the repository. +- Your project must use the [merge method](methods/index.md#fast-forward-merge) **Merge Commit**, + which is set in the project's **Settings > Merge requests**. Fast-forwarded commits + can't be cherry-picked from the GitLab UI, but the individual commits can + [still be cherry-picked](#cherry-pick-a-single-commit). + +To do this: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Code > Merge requests**, and find your merge request. diff --git a/doc/user/project/merge_requests/dependencies.md b/doc/user/project/merge_requests/dependencies.md index 89305e65dfb..8fb5230c497 100644 --- a/doc/user/project/merge_requests/dependencies.md +++ b/doc/user/project/merge_requests/dependencies.md @@ -145,6 +145,12 @@ information, read [issue #12549](https://gitlab.com/gitlab-org/gitlab/-/issues/1 ### Complex merge order dependencies are unsupported +- Support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11393) in GitLab 16.6 [with a flag](../../../administration/feature_flags.md) named `remove_mr_blocking_constraints`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `remove_mr_blocking_constraints`. +On GitLab.com, this feature is available. + If you attempt to create an indirect, nested dependency, GitLab shows the error message: - Dependencies failed to save: Dependency chains are not supported diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 85ebc75e61f..a3b1920e375 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -7,22 +7,19 @@ type: reference, concepts # Draft merge requests **(FREE ALL)** -If a merge request isn't ready to merge, potentially because of continued development -or open threads, you can prevent it from being accepted before you -[mark it as ready](#mark-merge-requests-as-ready). Flag it as a draft to disable -the **Merge** button until you remove the **Draft** flag: +If a merge request isn't ready to merge, you can block it from merging until you +[mark it as ready](#mark-merge-requests-as-ready). Merge requests marked as **Draft** +cannot merge until the **Draft** flag is removed, even if all other merge criteria are met: - + ## Mark merge requests as drafts -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32692) in GitLab 13.2, Work-In-Progress (WIP) merge requests were renamed to **Draft**. -> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/228685) all support for using **WIP** in GitLab 14.8. -> - **Mark as draft** and **Mark as ready** buttons [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227421) in GitLab 13.5. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/228685) all support for the term **WIP** in GitLab 14.8. > `/draft` quick action as a toggle [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) in GitLab 15.4. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108073) the draft status to use a checkbox in GitLab 15.8. -There are several ways to flag a merge request as a draft: +You can flag a merge request as a draft in several ways: - **Viewing a merge request**: In the upper-right corner of the merge request, select **Mark as draft**. - **Creating or editing a merge request**: Add `[Draft]`, `Draft:` or `(Draft)` to @@ -33,12 +30,12 @@ There are several ways to flag a merge request as a draft: in a comment. To mark a merge request as ready, use `/ready`. - **Creating a commit**: Add `draft:`, `Draft:`, `fixup!`, or `Fixup!` to the beginning of a commit message targeting the merge request's source branch. This - is not a toggle, and adding this text again in a later commit doesn't mark the + method is not a toggle. Adding this text again in a later commit doesn't mark the merge request as ready. ## Mark merge requests as ready -When a merge request is ready to be merged, you can remove the `Draft` flag in several ways: +When a merge request is ready to merge, you can remove the `Draft` flag in several ways: - **Viewing a merge request**: In the upper-right corner of the merge request, select **Mark as ready**. Users with at least the Developer role @@ -50,18 +47,18 @@ When a merge request is ready to be merged, you can remove the `Draft` flag in s [quick action](../quick_actions.md#issues-merge-requests-and-epics) in a comment in the merge request. -In [GitLab 13.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/15332), -when you mark a merge request as ready, notifications are triggered to -[merge request participants and watchers](../../profile/notifications.md#notifications-on-issues-merge-requests-and-epics). +When you mark a merge request as ready, +[merge request participants and watchers](../../profile/notifications.md#notifications-on-issues-merge-requests-and-epics) +are notified. ## Include or exclude drafts when searching -When viewing or searching in your project's merge requests list, you can include or exclude +When you view or search in your project's merge requests list, to include or exclude draft merge requests: 1. Go to your project and select **Code > Merge requests**. -1. In the navigation bar, select **Open**, **Merged**, **Closed**, or **All** to - filter by merge request status. +1. To filter by merge request status, select **Open**, **Merged**, **Closed**, + or **All** in the navigation bar. 1. Select the search box to display a list of filters and select **Draft**, or enter the word `draft`. 1. Select `=`. @@ -72,9 +69,9 @@ draft merge requests: ## Pipelines for drafts -Draft merge requests run the same pipelines as merge request that are marked as ready. +Draft merge requests run the same pipelines as merge requests marked as ready. -In GitLab 15.0 and older, you must [mark the merge request as ready](#mark-merge-requests-as-ready) +In GitLab 15.0 and earlier, you must [mark the merge request as ready](#mark-merge-requests-as-ready) if you want to run [merged results pipelines](../../../ci/pipelines/merged_results_pipelines.md). <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 22cd8f9b89e..63e5cc93e7d 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -82,6 +82,7 @@ or: > - Filtering by `reviewer` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47605) in GitLab 13.7. > - Filtering by potential approvers was moved to GitLab Premium in 13.9. > - Filtering by `approved-by` moved to GitLab Premium in 13.9. +> - Filtering by `source-branch` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134555) in GitLab 16.6. To filter the list of merge requests: @@ -489,3 +490,31 @@ p = Project.find_by_full_path('<namespace/project>') m = p.merge_requests.find_by(iid: <iid>) Issuable::DestroyService.new(container: m.project, current_user: u).execute(m) ``` + +### Merge request pre-receive hook failed + +If a merge request times out, you might see messages that indicate a Puma worker +timeout problem: + +- In the GitLab UI: + + ```plaintext + Something went wrong during merge pre-receive hook. + 500 Internal Server Error. Try again. + ``` + +- In the `gitlab-rails/api_json.log` log file: + + ```plaintext + Rack::Timeout::RequestTimeoutException + Request ran for longer than 60000ms + ``` + +This error can happen if your merge request: + +- Contains many diffs. +- Is many commits behind the target branch. + +Users in self-managed installations can request an administrator review server logs +to determine the cause of the error. GitLab SaaS users should +[contact Support](https://about.gitlab.com/support/#contact-support) for help. diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 699c79806f0..c4c38ef9eaf 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -79,7 +79,7 @@ merge. This configuration works for both: - GitLab CI/CD pipelines. - Pipelines run from an [external CI integration](../integrations/index.md#available-integrations). -As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md#disable-cicd-in-a-project) +As a result, [disabling GitLab CI/CD pipelines](../../../ci/pipelines/settings.md#disable-gitlab-cicd-pipelines) does not disable this feature, but you can use pipelines from external CI providers with it. diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 7e6bf606f10..4476ec8c670 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -25,7 +25,7 @@ Prerequisites: - You must have a role in the project that allows you to edit merge requests, and add code to the repository. - Your project must use the [merge method](methods/index.md#fast-forward-merge) **Merge Commit**, - which is set in the project's **Settings > General > Merge request**. You can't revert + which is set in the project's **Settings > Merge requests**. You can't revert fast-forwarded commits from the GitLab UI. To do this: diff --git a/doc/user/project/merge_requests/reviews/data_usage.md b/doc/user/project/merge_requests/reviews/data_usage.md index b4b9b19c932..b32c527ab75 100644 --- a/doc/user/project/merge_requests/reviews/data_usage.md +++ b/doc/user/project/merge_requests/reviews/data_usage.md @@ -13,7 +13,7 @@ GitLab Duo Suggested Reviewers is the first user-facing GitLab machine learning ### Enabling the feature -When a Project Maintainer or Owner enables Suggested Reviewers in project settings GitLab kicks off a data extraction job for the project which leverages the Merge Request API to understand pattern of review including recency, domain experience, and frequency to suggest an appropriate reviewer. +When a Project Maintainer or Owner enables Suggested Reviewers in project settings, GitLab kicks off a data extraction job for the project which leverages the Merge Request API to understand pattern of review including recency, domain experience, and frequency to suggest an appropriate reviewer. If projects do not use the [merge request approval process](../approvals/index.md) or do not have any historical merge request data, Suggested Reviewers cannot suggest reviewers. This data extraction job can take a few hours to complete (possibly up to a day), which is largely dependent on the size of the project. The process is automated and no action is needed during this process. Once data extraction is complete, you start getting suggestions in merge requests. diff --git a/doc/user/project/merge_requests/reviews/img/comment-on-any-diff-line_v13_10.png b/doc/user/project/merge_requests/reviews/img/comment-on-any-diff-line_v13_10.png Binary files differdeleted file mode 100644 index a31fea85be9..00000000000 --- a/doc/user/project/merge_requests/reviews/img/comment-on-any-diff-line_v13_10.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/comment_on_any_diff_line_v16_6.png b/doc/user/project/merge_requests/reviews/img/comment_on_any_diff_line_v16_6.png Binary files differnew file mode 100644 index 00000000000..5ed210ad8bb --- /dev/null +++ b/doc/user/project/merge_requests/reviews/img/comment_on_any_diff_line_v16_6.png diff --git a/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v15_3.png b/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v15_3.png Binary files differdeleted file mode 100644 index b73dbb50cd2..00000000000 --- a/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v15_3.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v16_6.png b/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v16_6.png Binary files differnew file mode 100644 index 00000000000..3e11440a71b --- /dev/null +++ b/doc/user/project/merge_requests/reviews/img/mr_review_new_comment_v16_6.png diff --git a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png Binary files differdeleted file mode 100644 index 47b7be3886d..00000000000 --- a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v15_4.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v16_6.png b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v16_6.png Binary files differnew file mode 100644 index 00000000000..965ce84a70f --- /dev/null +++ b/doc/user/project/merge_requests/reviews/img/mr_summary_comment_v16_6.png diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 0a3efa38440..d3124b716da 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -26,9 +26,13 @@ For an overview, see [Merge request review](https://www.youtube.com/watch?v=2May > - [Introduced](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/review-recommender/-/epics/3) in GitLab 15.4 as a [Beta](../../../../policy/experiment-beta-support.md#beta) feature [with a flag](../../../../administration/feature_flags.md) named `suggested_reviewers_control`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/368356) in GitLab 15.6. > - Beta designation [removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113058) in GitLab 15.10. +> - Feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134728) in GitLab 16.6. GitLab uses machine learning to suggest reviewers for your merge request. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [GitLab Duo Suggested Reviewers](https://www.youtube.com/embed/ivwZQgh4Rxw). + To suggest reviewers, GitLab uses: - The changes in the merge request @@ -164,7 +168,7 @@ You can submit your completed review in multiple ways: In the modal window, you can supply a **Summary comment**, approve the merge request, and include quick actions: -  +  When you submit your review, GitLab: @@ -193,7 +197,7 @@ Pending comments display information about the action to be taken when the comme If you have a review in progress, you can also add a comment from the **Overview** tab by selecting **Add to review**: - + ### Approval Rule information for Reviewers **(PREMIUM ALL)** @@ -227,8 +231,6 @@ them a notification email. When commenting on a diff, you can select which lines of code your comment refers to by either: - - - Dragging **Add a comment to this line** (**{comment}**) in the gutter to highlight lines in the diff. GitLab expands the diff lines and displays a comment box. - After starting a comment by selecting **Add a comment to this line** (**{comment}**) in the @@ -236,6 +238,8 @@ to by either: select box. New comments default to single-line comments, unless you select a different starting line. + + Multiline comments display the comment's line numbers above the body of the comment:  @@ -340,6 +344,9 @@ from the command line by running `git checkout <branch-name>`. ### Checkout merge requests locally through the `head` ref +> - Deleting `head` refs 14 days after a merge request closes or merges [enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/130098) in GitLab 16.4. +> - Deleting `head` refs 14 days after a merge request closes or merges [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/336070) in GitLab 16.6. Feature flag `merge_request_refs_cleanup` removed. + A merge request contains all the history from a repository, plus the additional commits added to the branch associated with the merge request. Here's a few ways to check out a merge request locally. @@ -351,9 +358,8 @@ This relies on the merge request `head` ref (`refs/merge-requests/:iid/head`) that is available for each merge request. It allows checking out a merge request by using its ID instead of its branch. -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223156) in GitLab -13.4, 14 days after a merge request gets closed or merged, the merge request -`head` ref is deleted. This means that the merge request isn't available +In GitLab 16.6 and later, the merge request `head` ref is deleted 14 days after +a merge request is closed or merged. The merge request is then no longer available for local checkout from the merge request `head` ref anymore. The merge request can still be re-opened. If the merge request's branch exists, you can still check out the branch, as it isn't affected. diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index 698078351e2..c330af0fc9b 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -10,6 +10,8 @@ type: reference, concepts > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3869) in GitLab 14.0, disabled behind the `:ff_external_status_checks` feature flag. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/320783) in GitLab 14.1. > - `failed` status [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329636) in GitLab 14.9. +> - `pending` status [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/413723) in GitLab 16.5 +> - Timeout interval of two minutes for `pending` status checks [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388725) in GitLab 16.6. Status checks are API calls to external systems that request the status of an external requirement. @@ -25,6 +27,8 @@ at the merge request level itself. You can configure merge request status checks for each individual project. These are not shared between projects. +Status checks fail if they stay in the pending state for more than two minutes. + For more information about use cases, feature discovery, and development timelines, see [epic 3869](https://gitlab.com/groups/gitlab-org/-/epics/3869). diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md index 8471a4ec55a..39d80517bc7 100644 --- a/doc/user/project/pages/public_folder.md +++ b/doc/user/project/pages/public_folder.md @@ -126,6 +126,15 @@ pages: NOTE: GitLab Pages supports only static sites. +By default, Nuxt uses the `public` folder to store static assets. For GitLab +Pages, rename the `public` folder to a collision-free alternative first: + +1. In your project directory, run: + + ```shell + mv public static + ``` + 1. Add the following to your `nuxt.config.js`: ```javascript @@ -133,6 +142,12 @@ GitLab Pages supports only static sites. target: 'static', generate: { dir: 'public' + }, + dir: { + // The folder name Nuxt uses for static files (`public`) is already + // reserved for the build output. So in deviation from the defaults we're + // using a folder called `static` instead. + public: 'static' } } ``` diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index fac07a1313a..f8f44d344d1 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -17,6 +17,7 @@ A protected branch controls: - If users can force push to the branch. - If changes to files listed in the CODEOWNERS file can be pushed directly to the branch. - Which users can unprotect the branch. +- Which users can modify the branch via the [Commits API](../../api/commits.md). The [default branch](repository/branches/default.md) for your repository is protected by default. @@ -26,12 +27,12 @@ The [default branch](repository/branches/default.md) for your repository is prot When a branch is protected, the default behavior enforces these restrictions on the branch. -| Action | Who can do it | -|:-------------------------|:------------------------------------------------------------------| -| Protect a branch | At least the Maintainer role. | +| Action | Who can do it | +|:-------------------------|:----------------------------------------| +| Protect a branch | At least the Maintainer role. | | Push to the branch | Anyone with **Allowed** permission. (1) | -| Force push to the branch | No one. (3) | -| Delete the branch | No one. (2) | +| Force push to the branch | No one. (3) | +| Delete the branch | No one. (2) | 1. Users with the Developer role can create a project in a group, but might not be allowed to initially push to the [default branch](repository/branches/default.md). @@ -49,12 +50,12 @@ level of protection for the branch. For example, consider these rules, which inc [wildcards](#protect-multiple-branches-with-wildcard-rules): | Branch name pattern | Allowed to merge | Allowed to push and merge | -|---------------------|------------------------|-----------------| -| `v1.x` | Maintainer | Maintainer | -| `v1.*` | Maintainer + Developer | Maintainer | -| `v*` | No one | No one | +|---------------------|------------------------|---------------------------| +| `v1.x` | Maintainer | Maintainer | +| `v1.*` | Maintainer + Developer | Maintainer | +| `v*` | No one | No one | -A branch named `v1.x` matches all three branch name patterns: `v1.x`, `v1.*`, and `v*`. +A branch named `v1.x` is a case-sensitive match for all three branch name patterns: `v1.x`, `v1.*`, and `v*`. As the most permissive option determines the behavior, the resulting permissions for branch `v1.x` are: - **Allowed to merge:** Of the three settings, `Maintainer + Developer` is most permissive, @@ -71,10 +72,10 @@ If you want to ensure that `No one` is allowed to push to branch `v1.x`, every p that matches `v1.x` must set `Allowed to push and merge` to `No one`, like this: | Branch name pattern | Allowed to merge | Allowed to push and merge | -|---------------------|------------------------|-----------------| -| `v1.x` | Maintainer | No one | -| `v1.*` | Maintainer + Developer | No one | -| `v*` | No one | No one | +|---------------------|------------------------|---------------------------| +| `v1.x` | Maintainer | No one | +| `v1.*` | Maintainer + Developer | No one | +| `v*` | No one | No one | ### Set the default branch protection level @@ -138,6 +139,7 @@ To protect a branch for all the projects in a group: 1. Expand **Protected branches**. 1. Select **Add protected branch**. 1. In the **Branch** text box, type the branch name or a wildcard. + Branch names and wildcards [are case-sensitive](repository/branches/index.md#name-your-branch). 1. From the **Allowed to merge** list, select a role that can merge into this branch. 1. From the **Allowed to push and merge** list, select a role that can push to this branch. 1. Select **Protect**. @@ -162,7 +164,7 @@ To protect multiple branches at the same time: 1. Expand **Protected branches**. 1. Select **Add protected branch**. 1. From the **Branch** dropdown list, type the branch name and a wildcard. - For example: + Branch names and wildcards [are case-sensitive](repository/branches/index.md#name-your-branch). For example: | Wildcard protected branch | Matching branches | |---------------------------|--------------------------------------------------------| @@ -370,6 +372,7 @@ branches by using the GitLab web interface: 1. Select **Code > Branches**. 1. Next to the branch you want to delete, select **Delete** (**{remove}**). 1. On the confirmation dialog, enter the branch name and select **Yes, delete protected branch**. + Branch names [are case-sensitive](repository/branches/index.md#name-your-branch). Protected branches can only be deleted by using GitLab either from the UI or API. This prevents accidentally deleting a branch through local Git commands or @@ -381,14 +384,10 @@ third-party Git clients. - [Branches](repository/branches/index.md) - [Branches API](../../api/branches.md) -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Branch names are case-sensitive -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +Branch names in `git` are case-sensitive. When configuring your protected branch +or [target branch rule](repository/branches/index.md#configure-rules-for-target-branches), +`dev` is not the same `DEV` or `Dev`. diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index e8451e3049d..6c89e09bd47 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -45,7 +45,8 @@ Git push options can perform actions for merge requests while pushing changes: | Push option | Description | |----------------------------------------------|-------------| | `merge_request.create` | Create a new merge request for the pushed branch. | -| `merge_request.target=<branch_name>` | Set the target of the merge request to a particular branch or upstream project, such as: `git push -o merge_request.target=project_path/branch`. | +| `merge_request.target=<branch_name>` | Set the target of the merge request to a particular branch, such as: `git push -o merge_request.target=branch_name`. | +| `merge_request.target_project=<project>` | Set the target of the merge request to a particular upstream project, such as: `git push -o merge_request.target_project=path/to/project`. Introduced in [GitLab 16.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/132475). | | `merge_request.merge_when_pipeline_succeeds` | Set the merge request to [merge when its pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md). | | `merge_request.remove_source_branch` | Set the merge request to remove the source branch when it's merged. | | `merge_request.title="<title>"` | Set the title of the merge request. For example: `git push -o merge_request.title="The title I want"`. | diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 30ddf8d3230..3640beebdfb 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -176,6 +176,7 @@ GitLab enforces these additional rules on all branches: - No spaces are allowed in branch names. - Branch names with 40 hexadecimal characters are prohibited, because they are similar to Git commit hashes. +- Branch names are case-sensitive. Common software packages, like Docker, can enforce [additional branch naming restrictions](../../../../administration/packages/container_registry.md#docker-connection-error). @@ -313,6 +314,27 @@ To create a target branch rule: 1. Select the **Target branch** to use when the branch name matches the **Rule name**. 1. Select **Save**. +### Example + +You could configure your project to have the following target branch rules: + +| Rule name | Target branch | +|-------------|---------------| +| `feature/*` | `develop` | +| `bug/*` | `develop` | +| `release/*` | `main` | + +These rules simplify the process of creating merge requests for a project that: + +- Uses `main` to represent the deployed state of your application. +- Tracks current, unreleased development work in another long-running branch, like `develop`. + +If your workflow initially places new features in `develop` instead of `main`, these rules +ensure all branches matching either `feature/*` or `bug/*` do not target `main` by mistake. + +When you're ready to release to `main`, create a branch named `release/*`, and the rules +ensure this branch targets `main`. + ## Delete a target branch rule When you remove a target branch rule, existing merge requests remain unchanged. @@ -389,3 +411,18 @@ To fix this problem: Git versions [2.16.0 and later](https://github.com/git/git/commit/a625b092cc59940521789fe8a3ff69c8d6b14eb2), prevent you from creating a branch with this name. + +### Find all branches you've authored + +To find all branches you've authored in a project, run this command in a Git repository: + +```shell +git for-each-ref --format='%(refname:short) %(authoremail)' | grep $(git config --get user.email) +``` + +To get a total of all branches in a project, sorted by author, run this command +in a Git repository: + +```shell +git for-each-ref --format='%(authoremail)' | sort | uniq -c | sort -g +``` diff --git a/doc/user/project/repository/code_suggestions/index.md b/doc/user/project/repository/code_suggestions/index.md index 151792089ce..b44e26f8daf 100644 --- a/doc/user/project/repository/code_suggestions/index.md +++ b/doc/user/project/repository/code_suggestions/index.md @@ -17,6 +17,11 @@ Beta users should read about the [known limitations](#known-limitations). We loo Write code more efficiently by using generative AI to suggest code while you're developing. +Code Suggestions supports two distinct types of interactions: + +- Code Completion, which suggests completions the current line you are typing. These suggestions are usually low latency. +- Code Generation, which generates code based on a natural language code comment block. Generating code can exceed multiple seconds. + GitLab Duo Code Suggestions are available: - On [self-managed](self_managed.md) and [SaaS](saas.md). @@ -31,7 +36,7 @@ GitLab Duo Code Suggestions are available: </figure> During Beta, usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). -Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). +Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). As Code Suggestions matures to General Availibility it will be governed by our [AI Functionality Terms](https://about.gitlab.com/handbook/legal/ai-functionality-terms/). ## Use Code Suggestions @@ -62,22 +67,13 @@ Code Suggestions do not prevent you from writing code in your IDE. ## Supported languages -The best results from Code Suggestions are expected for languages that [Anthropic Claude](https://www.anthropic.com/product) and the [Google Vertex AI Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_coding_languages) directly support: - -- C++ -- C# -- Go -- Google SQL -- Java -- JavaScript -- Kotlin -- PHP -- Python -- Ruby -- Rust -- Scala -- Swift -- TypeScript +Code Suggestions support is a function of the: + +- Underlying large language model. +- IDE used. +- Extension or plug-in support in the IDE. + +For languages not listed in the following table, Code Suggestions might not function as expected. ### Supported languages in IDEs @@ -129,10 +125,12 @@ This improvement should result in: Code Suggestions is powered by a generative AI model. Your personal access token enables a secure API connection to GitLab.com or to your GitLab instance. -This API connection securely transmits a context window from your IDE/editor to the [GitLab AI Gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist), a GitLab hosted service. The gateway calls the large language model APIs, and then the generated suggestion is transmitted back to your IDE/editor. +This API connection securely transmits a context window from your IDE/editor to the [GitLab AI Gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist), a GitLab hosted service. The [gateway](../../../../development/ai_architecture.md) calls the large language model APIs, and then the generated suggestion is transmitted back to your IDE/editor. GitLab selects the best-in-class large-language models for specific tasks. We use [Google Vertex AI Code Models](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) and [Anthropic Claude](https://www.anthropic.com/product) for Code Suggestions. +[View data retention policies](../../../ai_features.md#data-retention). + ### Telemetry For self-managed instances that have enabled Code Suggestions and SaaS accounts, we collect aggregated or de-identified first-party usage data through our [Snowplow collector](https://about.gitlab.com/handbook/business-technology/data-team/platform/snowplow/). This usage data includes the following metrics: diff --git a/doc/user/project/repository/code_suggestions/self_managed.md b/doc/user/project/repository/code_suggestions/self_managed.md index ee501212027..fd363e56021 100644 --- a/doc/user/project/repository/code_suggestions/self_managed.md +++ b/doc/user/project/repository/code_suggestions/self_managed.md @@ -164,7 +164,7 @@ A self-managed GitLab instance does not generate the code suggestion. After succ authentication to the self-managed instance, a token is generated. The IDE/editor then uses this token to securely transmit data directly to -GitLab.com's Code Suggestions service for processing. +GitLab.com's Code Suggestions service via the [Cloud Connector gateway service](../../../../architecture/blueprints/cloud_connector/index.md) for processing. The Code Suggestions service then securely returns an AI-generated code suggestion. diff --git a/doc/user/project/repository/code_suggestions/troubleshooting.md b/doc/user/project/repository/code_suggestions/troubleshooting.md index 2faf20b3035..86400ea8860 100644 --- a/doc/user/project/repository/code_suggestions/troubleshooting.md +++ b/doc/user/project/repository/code_suggestions/troubleshooting.md @@ -18,9 +18,6 @@ In GitLab, ensure Code Suggestions is enabled: - [For your user account](../../../profile/preferences.md#enable-code-suggestions). - [For *all* top-level groups your account belongs to](../../../group/manage.md#enable-code-suggestions). If you don't have a role that lets you view the top-level group's settings, contact a group owner. -To confirm that your account is enabled, go to [https://gitlab.com/api/v4/ml/ai-assist](https://gitlab.com/api/v4/ml/ai-assist). The `user_is_allowed` key should have should have a value of `true`. -A `404 Not Found` result is returned if either of the previous conditions is not met. - ### Code Suggestions not displayed in VS Code or GitLab WebIDE Check all the steps in [Code Suggestions aren't displayed](#code-suggestions-arent-displayed) first. diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index ddc650c3924..c71c89b68c3 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -24,17 +24,24 @@ can access the object pool connected to the source project. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) a new form in GitLab 13.11 [with a flag](../../../user/feature_flags.md) named `fork_project_form`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77181) in GitLab 14.8. Feature flag `fork_project_form` removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24894) in GitLab 16.6. To fork an existing project in GitLab: 1. On the project's homepage, in the upper-right corner, select **Fork** (**{fork}**): +  + 1. Optional. Edit the **Project name**. 1. For **Project URL**, select the [namespace](../../namespace/index.md) your fork should belong to. 1. Add a **Project slug**. This value becomes part of the URL to your fork. It must be unique in the namespace. 1. Optional. Add a **Project description**. +1. Select one of the **Branches to include** options: + - **All branches** (default). + - **Only the default branch**. Uses the `--single-branch` and `--no-tags` + [Git options](https://git-scm.com/docs/git-clone). 1. Select the **Visibility level** for your fork. For more information about visibility levels, read [Project and group visibility](../../public_access.md). 1. Select **Fork project**. diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index ff9ef5b78f8..ca7f2ae2043 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -325,12 +325,15 @@ are accurate. To expedite this process, see the ['Prune Unreachable Objects' housekeeping task](../../../administration/housekeeping.md). -### Sidekiq process fails to export a project +### Sidekiq process fails to export a project **(FREE SELF)** Occasionally the Sidekiq process can fail to export a project, for example if it is terminated during execution. -To bypass the Sidekiq process, use the Rails console to manually trigger the project export: +GitLab.com users should [contact Support](https://about.gitlab.com/support/#contact-support) to resolve this issue. + +Self-managed users can use the Rails console to bypass the Sidekiq process and +manually trigger the project export: ```ruby project = Project.find(1) diff --git a/doc/user/project/service_desk/configure.md b/doc/user/project/service_desk/configure.md index 172a105cc28..8d0fbd81ebd 100644 --- a/doc/user/project/service_desk/configure.md +++ b/doc/user/project/service_desk/configure.md @@ -191,6 +191,8 @@ The custom email address you want to use must meet all of the following requirem by any text to the local part. Given the email address `support@example.com`, check whether sub-addressing is supported by sending an email to `support+1@example.com`. This email should appear in your mailbox. - You have SMTP credentials (ideally, you should use an app password). + The username and password are stored in the database using the Advanced Encryption Standard (AES) + with a 256-bit key. - You must have at least the Maintainer role for the project. - Service Desk must be configured for the project. diff --git a/doc/user/project/service_desk/using_service_desk.md b/doc/user/project/service_desk/using_service_desk.md index ad97a36bbb0..5f3c725b83b 100644 --- a/doc/user/project/service_desk/using_service_desk.md +++ b/doc/user/project/service_desk/using_service_desk.md @@ -138,10 +138,7 @@ HTML emails show HTML formatting, such as: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 15.8 [with a flag](../../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386860) in GitLab 15.10. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 16.6. Feature flag `service_desk_new_note_email_native_attachments` removed. If a comment contains any attachments and their total size is less than or equal to 10 MB, these attachments are sent as part of the email. In other cases, the email contains links to the attachments. diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 7de8a7beab5..3526425c912 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -60,7 +60,7 @@ To create a project access token: 1. Enter a name. The token name is visible to any user with permissions to view the project. 1. Enter an expiry date for the token. - The token expires on that date at midnight UTC. - - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. + - If you do not enter an expiry date, the expiry date is automatically set to 30 days later than the current date. - By default, this date can be a maximum of 365 days later than the current date. - An instance-wide [maximum lifetime](../../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. diff --git a/doc/user/project/system_notes.md b/doc/user/project/system_notes.md index 73509846990..546b3250180 100644 --- a/doc/user/project/system_notes.md +++ b/doc/user/project/system_notes.md @@ -23,12 +23,14 @@ in system notes. System notes use the format `<Author> <action> <time ago>`. By default, system notes do not display. When displayed, they are shown oldest first. If you change the filter or sort options, your selection is remembered across sections. -The filtering options are: +For all item types except merge requests, the filtering options are: - **Show all activity** displays both comments and history. - **Show comments only** hides system notes. - **Show history only** hides user comments. +Merge requests provide more granular filtering options. + ### On an epic 1. On the left sidebar, select **Search or go to** and find your project. @@ -49,7 +51,19 @@ The filtering options are: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Code > Merge requests** and find your merge request. 1. Go to **Activity**. -1. For **Sort or filter**, select **Show all activity**. +1. For **Sort or filter**, select **Show all activity** to see all system notes. + To narrow the types of system notes returned, select one or more of: + + - **Approvals** + - **Assignees & Reviewers** + - **Comments** + - **Commits & branches** + - **Edits** + - **Labels** + - **Lock status** + - **Mentions** + - **Merge request status** + - **Tracking** ## Privacy considerations diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index a80c699eab7..fd543263ebd 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -181,11 +181,7 @@ You need at least the Developer role to move a wiki page: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414691) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `print_wiki`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134251/) in GitLab 16.5. - -FLAG: -On self-managed GitLab, by default this feature is available. -To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `print_wiki`. -On GitLab.com, this feature is available. +> - Feature flag `print_wiki` removed in GitLab 16.6. You can export a wiki page as a PDF file: diff --git a/doc/user/read_only_namespaces.md b/doc/user/read_only_namespaces.md index 5b302d976dd..d5697ec5a94 100644 --- a/doc/user/read_only_namespaces.md +++ b/doc/user/read_only_namespaces.md @@ -27,7 +27,7 @@ To restore a namespace to its standard state, you can: - [Purchase a paid tier](https://about.gitlab.com/pricing/). - For exceeded storage quota: - [Purchase more storage for the namespace](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer). - - [Manage your storage usage](usage_quotas.md#manage-your-storage-usage). + - [Manage your storage usage](usage_quotas.md#manage-storage-usage). ## Restricted actions diff --git a/doc/user/report_abuse.md b/doc/user/report_abuse.md index 45113562e87..9e13d1fe263 100644 --- a/doc/user/report_abuse.md +++ b/doc/user/report_abuse.md @@ -26,17 +26,12 @@ You can report a user through their: > - Report abuse from overflow menu [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414773) in GitLab 16.4 [with a flag](../administration/feature_flags.md) named `user_profile_overflow_menu_vue`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/414773) in GitLab 16.4. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `user_profile_overflow_menu_vue`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/414773) in GitLab 16.6. Feature flag `user_profile_overflow_menu_vue` removed. To report abuse from a user's profile page: 1. Anywhere in GitLab, select the name of the user. -1. In the upper-right corner of the user's profile, if the `user_profile_overflow_menu_vue` feature flag is: - - Enabled, select the vertical ellipsis (**{ellipsis_v}**), then **Report abuse to administrator**. - - Disabled, select **Report abuse to administrator** (**{information-o}**). +1. In the upper-right corner of the user's profile select the vertical ellipsis (**{ellipsis_v}**), then **Report abuse to administrator**. 1. Select a reason for reporting the user. 1. Complete an abuse report. 1. Select **Send report**. diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index b9c64739de0..697f5711396 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -6,31 +6,30 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reserved project and group names **(FREE ALL)** -Not all project & group names are allowed because they would conflict with -existing routes used by GitLab. +To not conflict with existing routes used by GitLab, some words cannot be used as project or group names. +These words are listed in the +[`path_regex.rb` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/path_regex.rb), +where: -For a list of words that are not allowed to be used as group or project names, see the -[`path_regex.rb` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/path_regex.rb) -under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists: - -- `TOP_LEVEL_ROUTES`: are names that are reserved as usernames or top level groups -- `PROJECT_WILDCARD_ROUTES`: are names that are reserved for child groups or projects. -- `GROUP_ROUTES`: are names that are reserved for all groups or projects. +- `TOP_LEVEL_ROUTES` are names reserved as usernames or top-level groups. +- `PROJECT_WILDCARD_ROUTES` are names reserved for child groups or projects. +- `GROUP_ROUTES` are names reserved for all groups or projects. ## Limitations on project and group names -- Project or group names must start with a letter, digit, emoji, or "_". -- Project names can only contain letters, digits, emoji, "_", ".", "+", dashes, or spaces. -- Group names can only contain letters, digits, emoji, "_", ".", parenthesis, dashes, or spaces. -- Project or group slugs must start with a letter or digit. -- Project or group slugs can only contain letters, digits, '_', '.', or dashes. -- Project or group slugs must not contain consecutive special characters. -- Project or group slugs cannot start or end with a special character. -- Project or group slugs cannot end in `.git` or `.atom`. +- Project or group names must start with a letter (`a-zA-Z`), digit (`0-9`), emoji, or underscore (`_`). Additionally: + - Project names can contain only letters (`a-zA-Z`), digits (`0-9`), emoji, underscores (`_`), dots (`.`), pluses (`+`), dashes (`-`), or spaces. + - Group names can contain only letters (`a-zA-Z`), digits (`0-9`), emoji, underscores (`_`), dots (`.`), parentheses (`()`), dashes (`-`), or spaces. +- Project or group slugs: + - Must start with a letter (`a-zA-Z`) or digit (`0-9`). + - Must not contain consecutive special characters. + - Cannot start or end with a special character. + - Cannot end in `.git` or `.atom`. + - Can contain only letters (`a-zA-Z`), digits (`0-9`), underscores (`_`), dots (`.`), or dashes (`-`). ## Reserved project names -It is not possible to create a project with the following names: +You cannot create projects with the following names: - `\-` - `badges` @@ -56,7 +55,7 @@ It is not possible to create a project with the following names: ## Reserved group names -The following names are reserved as top level groups: +You cannot create groups with the following names, because they are reserved for top-level groups: - `\-` - `.well-known` @@ -98,6 +97,6 @@ The following names are reserved as top level groups: - `users` - `v2` -These group names are unavailable as subgroup names: +You cannot create subgroups with the following names: - `\-` diff --git a/doc/user/search/index.md b/doc/user/search/index.md index e8dfbfa675a..79782b1c880 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -103,29 +103,15 @@ For example: ## Include archived projects in search results -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121981) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `search_projects_hide_archived`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/413821) in GitLab 16.3. Feature flag `search_projects_hide_archived` removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121981) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `search_projects_hide_archived` for project search. Disabled by default. +> - [Generally available](https://gitlab.com/groups/gitlab-org/-/epics/10957) in GitLab 16.6 for all search scopes. By default, archived projects are excluded from search results. -To include archived projects: +To include archived projects in search results: -1. On the project search page, on the left sidebar, select the **Include archived** checkbox. +1. On the search page, on the left sidebar, select the **Include archived** checkbox. 1. On the left sidebar, select **Apply**. -## Exclude issues in archived projects from search results - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124846) in GitLab 16.2 [with a flag](../../administration/feature_flags.md) named `search_issues_hide_archived_projects`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -an administrator can [enable the feature flag](../../administration/feature_flags.md) named `search_issues_hide_archived_projects`. On GitLab.com, this feature is not available. - -By default, issues in archived projects are included in search results. -To exclude issues in archived projects, ensure the `search_issues_hide_archived_projects` flag is enabled. - -To include issues in archived projects with `search_issues_hide_archived_projects` enabled, -you must add the parameter `include_archived=true` to the URL. - ## Search for code To search for code in a project: diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index fa03cb54ba3..e504ee90821 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -1,6 +1,6 @@ --- -stage: none -group: unassigned +stage: Manage +group: Foundations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -135,6 +135,7 @@ These shortcuts are available when browsing the files in a project (go to | <kbd>Enter</kbd> | Open selection. | | <kbd>Escape</kbd> | Go back to file list screen (only while searching for files, **Code > Repository**, then select **Find File**). | | <kbd>y</kbd> | Go to file permalink (only while viewing a file). | +| <kbd>Shift</kbd> + <kbd>c</kbd> | Go to compare branches view. | | <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | ### Web IDE diff --git a/doc/user/snippets.md b/doc/user/snippets.md index dbcc90c26df..fe782227701 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -17,7 +17,7 @@ and you can maintain your snippets with the [snippets API](../api/snippets.md). You can create and manage your snippets through the GitLab user interface, or by using the [GitLab Workflow VS Code extension](project/repository/vscode.md). - + GitLab provides two types of snippets: @@ -168,10 +168,11 @@ To delete a file from your snippet through the GitLab UI: ## Clone snippets To ensure you receive updates, clone the snippet instead of copying it locally. Cloning -maintains the snippet's connection with the repository. Select **Clone** on a snippet -to display the URLs to clone with SSH or HTTPS: +maintains the snippet's connection with the repository. - +To clone a snippet: + +- Select **Clone**, then copy the URL to clone with SSH or HTTPS. You can commit changes to a cloned snippet, and push the changes to GitLab. diff --git a/doc/user/storage_management_automation.md b/doc/user/storage_management_automation.md index 96f9ecd11a8..a83af4ab6c6 100644 --- a/doc/user/storage_management_automation.md +++ b/doc/user/storage_management_automation.md @@ -14,6 +14,10 @@ You can also manage your storage usage by improving [pipeline efficiency](../ci/ For more help with API automation, you can also use the [GitLab community forum and Discord](https://about.gitlab.com/community/). +WARNING: +The script examples in this page are for demonstration purposes only and should not +be used in production. You can use the examples to design and test your own scripts for storage automation. + ## API requirements To automate storage management, your GitLab.com SaaS or self-managed instance must have access to the [GitLab REST API](../api/api_resources.md). @@ -567,11 +571,17 @@ Support for creating a retention policy for job logs is proposed in [issue 37471 ### Delete old pipelines -Pipelines do not add to the overall storage consumption, but if required you can delete them with the following methods. +Pipelines do not add to the overall storage usage, but if required you can automate their deletion. -Automatic deletion of old pipelines is proposed in [issue 338480](https://gitlab.com/gitlab-org/gitlab/-/issues/338480). +To delete pipelines based on a specific date, specify the `created_at` key. +You can use the date to calculate the difference between the current date and +when the pipeline was created. If the age is larger than the threshold, the pipeline is deleted. -Example with the GitLab CLI: +NOTE: +The `created_at` key must be converted from a timestamp to Unix epoch time, +for example with `date -d '2023-08-08T18:59:47.581Z' +%s`. + +Example with GitLab CLI: ```shell export GL_PROJECT_ID=48349590 @@ -589,12 +599,10 @@ glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '. "2023-08-08T18:59:47.581Z" ``` -The `created_at` key must be converted from a timestamp to Unix epoch time, -for example with `date -d '2023-08-08T18:59:47.581Z' +%s`. In the next step, the -age can be calculated with the difference between now, and the pipeline creation -date. If the age is larger than the threshold, the pipeline should be deleted. +In the following example that uses a Bash script: -The following example uses a Bash script that expects `jq` and the GitLab CLI installed, and authorized, and the exported environment variable `GL_PROJECT_ID`. +- `jq` and the GitLab CLI are installed and authorized. +- The exported environment variable `GL_PROJECT_ID`. The full script `get_cicd_pipelines_compare_age_threshold_example.sh` is located in the [GitLab API with Linux Shell](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-linux-shell) project. @@ -624,7 +632,7 @@ do done ``` -You can use the [`python-gitlab` API library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#project-pipelines) and +You can also use the [`python-gitlab` API library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#project-pipelines) and the `created_at` attribute to implement a similar algorithm that compares the job artifact age: ```python @@ -645,6 +653,8 @@ the `created_at` attribute to implement a similar algorithm that compares the jo pipeline_obj.delete() ``` +Automatic deletion of old pipelines is proposed in [issue 338480](https://gitlab.com/gitlab-org/gitlab/-/issues/338480). + ### List expiry settings for job artifacts To manage artifact storage, you can update or configure when an artifact expires. @@ -770,7 +780,7 @@ default: ## Manage Container Registries storage -Container registries are available [in a project](../api/container_registry.md#within-a-project) or [in a group](../api/container_registry.md#within-a-group). You can analyze both locations to implement a cleanup strategy. +Container registries are available [for projects](../api/container_registry.md#within-a-project) or [for groups](../api/container_registry.md#within-a-group). You can analyze both locations to implement a cleanup strategy. ### List container registries @@ -818,8 +828,6 @@ glab api --method GET projects/$GL_PROJECT_ID/registry/repositories/4435617/tags ::EndTabs -A similar automation shell script is created in the [delete old pipelines](#delete-old-pipelines) section. - ### Delete container images in bulk When you [delete container image tags in bulk](../api/container_registry.md#delete-registry-repository-tags-in-bulk), @@ -886,7 +894,7 @@ You can optimize container images to reduce the image size and overall storage c ## Manage Package Registry storage -Package registries are available [in a project](../api/packages.md#within-a-project) or [in a group](../api/packages.md#within-a-group). +Package registries are available [for projects](../api/packages.md#for-a-project) or [for groups](../api/packages.md#for-a-group). ### List packages and files diff --git a/doc/user/tasks.md b/doc/user/tasks.md index 347aedd6e74..173d2e44cf1 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -360,6 +360,25 @@ To copy the task's email address: 1. Select **Plan > Issues**, then select your issue to view it. 1. In the top right corner, select the vertical ellipsis (**{ellipsis_v}**), then select **Copy task email address**. +## Set an issue as a parent + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/11198) in GitLab 16.5. + +Prerequisite: + +- You must have at least the Reporter role for the project. +- The issue and task must belong to the same project. + +To set an issue as a parent of a task: + +1. In the issue description, in the **Tasks** section, select the title of the task you want to edit. + The task window opens. +1. Next to **Parent**, from the dropdown list, select the parent to add. +1. Select any area outside the dropdown list. + +To remove the parent item of the task, +next to **Parent**, select the dropdown list and then select **Unassign**. + ## Confidential tasks > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8410) in GitLab 15.3. diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 305a46e1f15..7dea2b97249 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -5,7 +5,7 @@ group: Utilization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Storage usage quota **(FREE ALL)** +# Storage **(FREE ALL)** Storage usage statistics are available for projects and namespaces. You can use that information to manage storage usage within the applicable quotas. @@ -13,8 +13,8 @@ manage storage usage within the applicable quotas. Statistics include: - Storage usage across projects in a namespace. -- Storage usage that exceeds the storage quota. -- Available purchased storage. +- Storage usage that exceeds the storage SaaS limit or [self-managed storage quota](../administration/settings/account_and_limit_settings.md#repository-size-limit). +- Available purchased storage for SaaS. Storage and network usage are calculated with the binary measurement system (1024 unit multiples). Storage usage is displayed in kibibytes (KiB), mebibytes (MiB), @@ -30,87 +30,33 @@ you might see references to `KB`, `MB`, and `GB` in the UI and documentation. Prerequisites: - To view storage usage for a project, you must have at least the Maintainer role for the project or Owner role for the namespace. -- To view storage usage for a namespace, you must have the Owner role for the namespace. +- To view storage usage for a group namespace, you must have the Owner role for the namespace. 1. On the left sidebar, select **Search or go to** and find your project or group. 1. On the left sidebar, select **Settings > Usage Quotas**. -1. Select the **Storage** tab. +1. Select the **Storage** tab to see namespace storage usage. +1. To view storage usage for a project, select one of the projects from the table at the bottom of the **Storage** tab of the **Usage Quotas** page. -Select any title to view details. The information on this page -is updated every 90 minutes. +The information on the **Usage Quotas** page is updated every 90 minutes. If your namespace shows `'Not applicable.'`, push a commit to any project in the namespace to recalculate the storage. -### Container Registry usage **(FREE SAAS)** +### View project fork storage usage **(FREE SAAS)** -Container Registry usage is available only for GitLab.com. This feature requires a -[new version](https://about.gitlab.com/blog/2022/04/12/next-generation-container-registry/) -of the GitLab Container Registry. To learn about the proposed release for self-managed -installations, see [epic 5521](https://gitlab.com/groups/gitlab-org/-/epics/5521). - -#### How container registry usage is calculated - -Image layers stored in the Container Registry are deduplicated at the root namespace level. - -An image is only counted once if: - -- You tag the same image more than once in the same repository. -- You tag the same image across distinct repositories under the same root namespace. - -An image layer is only counted once if: - -- You share the image layer across multiple images in the same container repository, project, or group. -- You share the image layer across different repositories. - -Only layers that are referenced by tagged images are accounted for. Untagged images and any layers -referenced exclusively by them are subject to [online garbage collection](packages/container_registry/delete_container_registry_images.md#garbage-collection). -Untagged image layers are automatically deleted after 24 hours if they remain unreferenced during that period. - -Image layers are stored on the storage backend in the original (usually compressed) format. This -means that the measured size for any given image layer should match the size displayed on the -corresponding [image manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md#example-image-manifest). - -Namespace usage is refreshed a few minutes after a tag is pushed or deleted from any container repository under the namespace. - -#### Delayed refresh - -It is not possible to calculate [container registry usage](#container-registry-usage) -with maximum precision in real time for extremely large namespaces (about 1% of namespaces). -To enable maintainers of these namespaces to see their usage, there is a delayed fallback mechanism. -See [epic 9413](https://gitlab.com/groups/gitlab-org/-/epics/9413) for more details. - -If the usage for a namespace cannot be calculated with precision, GitLab falls back to the delayed method. -In the delayed method, the displayed usage size is the sum of **all** unique image layers -in the namespace. Untagged image layers are not ignored. As a result, -the displayed usage size might not change significantly after deleting tags. Instead, -the size value only changes when: - -- An automated [garbage collection process](packages/container_registry/delete_container_registry_images.md#garbage-collection) - runs and deletes untagged image layers. After a user deletes a tag, a garbage collection run - is scheduled to start 24 hours later. During that run, images that were previously tagged - are analyzed and their layers deleted if not referenced by any other tagged image. - If any layers are deleted, the namespace usage is updated. -- The namespace's registry usage shrinks enough that GitLab can measure it with maximum precision. - As usage for namespaces shrinks to be under the [limits](#namespace-storage-limit), - the measurement switches automatically from delayed to precise usage measurement. - There is no place in the UI to determine which measurement method is being used, - but [issue 386468](https://gitlab.com/gitlab-org/gitlab/-/issues/386468) proposes to improve this. +A cost factor is applied to the storage consumed by project forks so that forks consume less namespace storage than their actual size. -### Storage usage statistics +To view the amount of namespace storage the fork has used: -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68898) project-level graph in GitLab 14.4 [with a flag](../administration/feature_flags.md) named `project_storage_ui`. Disabled by default. -> - Enabled on GitLab.com in GitLab 14.4. -> - Enabled on self-managed in GitLab 14.5. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71270) in GitLab 14.5. +1. On the left sidebar, select **Search or go to** and find your project or group. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select the **Storage** tab. The **Total** column displays the amount of namespace storage used by the fork as a portion of the actual size of the fork on disk. -The following storage usage statistics are available to a maintainer: +The cost factor applies to the project repository, LFS objects, job artifacts, packages, snippets, and the wiki. -- Total namespace storage used: Total amount of storage used across projects in this namespace. -- Total excess storage used: Total amount of storage used that exceeds their allocated storage. -- Purchased storage available: Total storage that has been purchased but is not yet used. +The cost factor does not apply to private forks in namespaces on the Free plan. -## Manage your storage usage +## Manage storage usage To manage your storage, if you are a namespace Owner you can [purchase more storage for the namespace](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer). @@ -126,14 +72,16 @@ Depending on your role, you can also use the following methods to manage or redu To automate storage usage analysis and management, see the [storage management automation](storage_management_automation.md) documentation. -## Manage your transfer usage +## Set usage quotas **(FREE SELF)** + +There are no application limits on the amount of storage and transfer for self-managed instances. The administrators are responsible for the underlying infrastructure costs. Administrators can set [repository size limits](../administration/settings/account_and_limit_settings.md#repository-size-limit) to manage your repositories’ size. -Depending on your role, to manage your transfer usage you can [reduce Container Registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md). +## Storage limits **(FREE SAAS)** -## Project storage limit +### Project storage limit -Projects on GitLab SaaS have a 10 GiB storage limit on their Git repository and LFS storage. -After namespace-level storage limits are applied, the project limit is removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both. +Projects on GitLab SaaS have a 10 GiB storage limit on their Git repository and LFS storage. Limits on project storage +will be removed before limits are applied to GitLab SaaS namespace storage in the future. When a project's repository and LFS reaches the quota, the project is set to a read-only state. You cannot push changes to a read-only project. To monitor the size of each @@ -141,7 +89,7 @@ repository in a namespace, including a breakdown for each project, [view storage usage](#view-storage-usage). To allow a project's repository and LFS to exceed the free quota you must purchase additional storage. For more details, see [Excess storage usage](#excess-storage-usage). -### Excess storage usage +#### Excess storage usage Excess storage usage is the amount that a project's repository and LFS exceeds the [project storage limit](#project-storage-limit). If no purchased storage is available the project is set to a read-only state. You cannot push changes to a read-only project. @@ -185,12 +133,19 @@ available decreases. All projects no longer have the read-only status because 40 | Yellow | 5 GiB | 0 GiB | 10 GiB | Not read-only | | **Totals** | **45 GiB** | **10 GiB** | - | - | -## Namespace storage limit +### Namespace storage limit **(FREE SAAS)** -Namespaces on GitLab SaaS have a storage limit. For more information, see our [pricing page](https://about.gitlab.com/pricing/). +GitLab plans to enforce a storage limit for namespaces on GitLab SaaS. For more information, see +the FAQs for the following tiers: -After namespace storage limits are enforced, view them in the **Usage quotas** page. -For more information about the namespace storage limit enforcement, see the FAQ pages for the [Free](https://about.gitlab.com/pricing/faq-efficient-free-tier/#storage-limits-on-gitlab-saas-free-tier) and [Paid](https://about.gitlab.com/pricing/faq-paid-storage-transfer/) tiers. +- [Free tier](https://about.gitlab.com/pricing/faq-efficient-free-tier/#storage-limits-on-gitlab-saas-free-tier). +- [Premium and Ultimate](https://about.gitlab.com/pricing/faq-paid-storage-transfer/). + +Namespaces on GitLab SaaS have a [10 GiB project limit](#project-storage-limit) with a soft limit on +namespace storage. Soft storage limits are limits that have not yet been enforced by GitLab, and will become +hard limits after namespace storage limits apply. To avoid your namespace from becoming +[read-only](../user/read_only_namespaces.md) after namespace storage limits apply, +you should ensure that your namespace storage adheres to the soft storage limit. Namespace storage limits do not apply to self-managed deployments, but administrators can [manage the repository size](../administration/settings/account_and_limit_settings.md#repository-size-limit). @@ -209,13 +164,13 @@ If your total namespace storage exceeds the available namespace storage quota, a To notify you that you have nearly exceeded your namespace storage quota: -- In the command-line interface, a notification displays after each `git push` action when you've reached 95% and 100% of your namespace storage quota. -- In the GitLab UI, a notification displays when you've reached 75%, 95%, and 100% of your namespace storage quota. +- In the command-line interface, a notification displays after each `git push` action when your namespace has reached between 95% and 100%+ of your namespace storage quota. +- In the GitLab UI, a notification displays when your namespace has reached between 75% and 100%+ of your namespace storage quota. - GitLab sends an email to members with the Owner role to notify them when namespace storage usage is at 70%, 85%, 95%, and 100%. To prevent exceeding the namespace storage limit, you can: -- [Manage your storage usage](#manage-your-storage-usage). +- [Manage your storage usage](#manage-storage-usage). - If you meet the eligibility requirements, you can apply for: - [GitLab for Education](https://about.gitlab.com/solutions/education/join/) - [GitLab for Open Source](https://about.gitlab.com/solutions/open-source/join/) @@ -225,16 +180,8 @@ To prevent exceeding the namespace storage limit, you can: - [Start a trial](https://about.gitlab.com/free-trial/) or [upgrade to GitLab Premium or Ultimate](https://about.gitlab.com/pricing/), which include higher limits and features to enable growing teams to ship faster without sacrificing on quality. - [Talk to an expert](https://page.gitlab.com/usage_limits_help.html) for more information about your options. -### View project fork storage usage - -A cost factor is applied to the storage consumed by project forks so that forks consume less namespace storage than their actual size. - -To view the amount of namespace storage the fork has used: - -1. On the left sidebar, select **Search or go to** and find your project or group. -1. On the left sidebar, select **Settings > Usage Quotas**. -1. Select the **Storage** tab. The **Total** column displays the amount of namespace storage used by the fork as a portion of the actual size of the fork on disk. - -The cost factor applies to the project repository, LFS objects, job artifacts, packages, snippets, and the wiki. +## Related Topics -The cost factor does not apply to private forks in namespaces on the Free plan. +- [Automate storage management](storage_management_automation.md) +- [Purchase storage and transfer](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) +- [Transfer usage](packages/container_registry/reduce_container_registry_data_transfer.md) diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md index 1284067a391..21905381577 100644 --- a/doc/user/workspace/index.md +++ b/doc/user/workspace/index.md @@ -95,18 +95,28 @@ Only these properties are relevant to the GitLab implementation of the `containe | `endpoints` | Port mappings to expose from the container. | | `volumeMounts` | Storage volume to mount in the container. | +### Using variables in a devfile + +You can define variables to use in your devfile. +The `variables` object is a map of name-value pairs that you can use for string replacement in the devfile. + +Variables cannot have names that start with `gl-`, `gl_`, `GL-`, or `GL_`. +For more information about how and where to use variables, see the [devfile documentation](https://devfile.io/docs/2.2.0/defining-variables). + ### Example configurations The following is an example devfile configuration: ```yaml schemaVersion: 2.2.0 +variables: + registry-root: registry.gitlab.com components: - name: tooling-container attributes: gl/inject-editor: true container: - image: registry.gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/debian-bullseye-ruby-3.2-node-18.12:rubygems-3.4-git-2.33-lfs-2.9-yarn-1.22-graphicsmagick-1.3.36-gitlab-workspaces + image: "{{registry-root}}/gitlab-org/remote-development/gitlab-remote-development-docs/debian-bullseye-ruby-3.2-node-18.12:rubygems-3.4-git-2.33-lfs-2.9-yarn-1.22-graphicsmagick-1.3.36-gitlab-workspaces" env: - name: KEY value: VALUE |
