diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-10-19 15:57:54 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-10-19 15:57:54 +0300 |
commit | 419c53ec62de6e97a517abd5fdd4cbde3a942a34 (patch) | |
tree | 1f43a548b46bca8a5fb8fe0c31cef1883d49c5b6 /doc/user | |
parent | 1da20d9135b3ad9e75e65b028bffc921aaf8deb7 (diff) |
Add latest changes from gitlab-org/gitlab@16-5-stable-eev16.5.0-rc42
Diffstat (limited to 'doc/user')
130 files changed, 2950 insertions, 1804 deletions
diff --git a/doc/user/ai_features.md b/doc/user/ai_features.md index feea06666dc..e24d50efee1 100644 --- a/doc/user/ai_features.md +++ b/doc/user/ai_features.md @@ -11,37 +11,48 @@ GitLab is creating AI-assisted features across our DevSecOps platform. These fea | Feature | Purpose | Large Language Model | Current availability | Maturity | |-|-|-|-|-| -| [Suggested Reviewers](project/merge_requests/reviews/index.md#gitlab-duo-suggested-reviewers) | Assists in creating faster and higher-quality reviews by automatically suggesting reviewers for your merge request. | GitLab creates a machine learning model for each project, which is used to generate reviewers <br><br> [View the issue](https://gitlab.com/gitlab-org/modelops/applied-ml/applied-ml-updates/-/issues/10) | SaaS only | [Generally Available (GA)](../policy/experiment-beta-support.md#generally-available-ga) | -| [Code Suggestions](project/repository/code_suggestions/index.md) | Helps you write code more efficiently by viewing code suggestions as you type. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS <br> Self-managed | [Beta](../policy/experiment-beta-support.md#beta) | -| [Vulnerability summary](application_security/vulnerabilities/index.md#explaining-a-vulnerability) | Helps you remediate vulnerabilities more efficiently, uplevel your skills, and write more secure code. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) <br><br> Anthropic's claude model if degraded performance | SaaS only <br><br> Ultimate tier | [Beta](../policy/experiment-beta-support.md#beta) | -| [Code explanation](#explain-code-in-the-web-ui-with-code-explanation) | Helps you understand code by explaining it in English language. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Chat](#answer-questions-with-chat) | Process and generate text and code in a conversational manner. Helps you quickly identify useful information in large volumes of text in issues, epics, code, and GitLab documentation. | Anthropic's claude model <br><br> OpenAI Embeddings | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Value stream forecasting](#forecast-deployment-frequency-with-value-stream-forecasting) | Assists you with predicting productivity metrics and identifying anomalies across your software development lifecycle. | Statistical forecasting | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Discussion summary](#summarize-issue-discussions-with-discussion-summary) | Assists with quickly getting everyone up to speed on lengthy conversations to help ensure you are all on the same page. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Merge request summary](project/merge_requests/ai_in_merge_requests.md#summarize-merge-request-changes) | Efficiently communicate the impact of your merge request changes. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Code review summary](project/merge_requests/ai_in_merge_requests.md#summarize-my-merge-request-review) | Helps ease merge request handoff between authors and reviewers and help reviewers efficiently understand suggestions. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Merge request template population](project/merge_requests/ai_in_merge_requests.md#fill-in-merge-request-templates) | Generate a description for the merge request based on the contents of the template. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Test generation](project/merge_requests/ai_in_merge_requests.md#generate-suggested-tests-in-merge-requests) | Automates repetitive tasks and helps catch bugs early. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Git suggestions](https://gitlab.com/gitlab-org/gitlab/-/issues/409636) | Helps you discover or recall Git commands when and where you need them. | OpenAI | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| **Root cause analysis** | Assists you in determining the root cause for a pipeline failure and failed CI/CD build. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | -| [Issue description generation](#summarize-an-issue-with-issue-description-generation) | Generate issue descriptions. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Suggested Reviewers](project/merge_requests/reviews/index.md#gitlab-duo-suggested-reviewers) | Assists in creating faster and higher-quality reviews by automatically suggesting reviewers for your merge request. | GitLab creates a machine learning model for each project, which is used to generate reviewers <br><br> [View the issue](https://gitlab.com/gitlab-org/modelops/applied-ml/applied-ml-updates/-/issues/10) | SaaS only <br><br> Ultimate tier | [Generally Available (GA)](../policy/experiment-beta-support.md#generally-available-ga) | +| [Code Suggestions](project/repository/code_suggestions/index.md) | Helps you write code more efficiently by viewing code suggestions as you type. | [`code-gecko`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-completion) and [`code-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-generation) <br><br> [Anthropic's Claude](https://www.anthropic.com/product) model | SaaS <br> Self-managed <br><br> All tiers | [Beta](../policy/experiment-beta-support.md#beta) | +| [Vulnerability summary](application_security/vulnerabilities/index.md#explaining-a-vulnerability) | Helps you remediate vulnerabilities more efficiently, uplevel your skills, and write more secure code. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) <br><br> Anthropic's claude model if degraded performance | SaaS only <br><br> Ultimate tier | [Beta](../policy/experiment-beta-support.md#beta) | +| [Code explanation](#explain-code-in-the-web-ui-with-code-explanation) | Helps you understand code by explaining it in English language. | [`codechat-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-chat) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [GitLab Duo Chat](#answer-questions-with-gitlab-duo-chat) | Process and generate text and code in a conversational manner. Helps you quickly identify useful information in large volumes of text in issues, epics, code, and GitLab documentation. | Anthropic's claude model <br><br> OpenAI Embeddings | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Value stream forecasting](#forecast-deployment-frequency-with-value-stream-forecasting) | Assists you with predicting productivity metrics and identifying anomalies across your software development lifecycle. | Statistical forecasting | SaaS only <br> Self-managed <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Discussion summary](#summarize-issue-discussions-with-discussion-summary) | Assists with quickly getting everyone up to speed on lengthy conversations to help ensure you are all on the same page. | OpenAI's GPT-3 | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Merge request summary](project/merge_requests/ai_in_merge_requests.md#summarize-merge-request-changes) | Efficiently communicate the impact of your merge request changes. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Code review summary](project/merge_requests/ai_in_merge_requests.md#summarize-my-merge-request-review) | Helps ease merge request handoff between authors and reviewers and help reviewers efficiently understand suggestions. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Merge request template population](project/merge_requests/ai_in_merge_requests.md#fill-in-merge-request-templates) | Generate a description for the merge request based on the contents of the template. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Test generation](project/merge_requests/ai_in_merge_requests.md#generate-suggested-tests-in-merge-requests) | Automates repetitive tasks and helps catch bugs early. | [`text-bison`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Git suggestions](https://gitlab.com/gitlab-org/gitlab/-/issues/409636) | Helps you discover or recall Git commands when and where you need them. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Root cause analysis](#root-cause-analysis) | Assists you in determining the root cause for a pipeline failure and failed CI/CD build. | [Google Vertex Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | +| [Issue description generation](#summarize-an-issue-with-issue-description-generation) | Generate issue descriptions. | OpenAI's GPT-3 | SaaS only <br><br> Ultimate tier | [Experiment](../policy/experiment-beta-support.md#experiment) | ## Enable AI/ML features -The [Generally Available](../policy/experiment-beta-support.md#generally-available-ga) features listed in the previous table do not need to be enabled. - -[Experiment features](../policy/experiment-beta-support.md#experiment) and [Beta features](../policy/experiment-beta-support.md#beta) (besides Code Suggestions) on SaaS must be enabled by a user who has the Owner role in the group. Their usage is subject to the [Testing Terms of Use](https://about.gitlab.com/handbook/legal/testing-agreement/). - -In addition, all features built on large language models (LLM) from Google, Anthropic or OpenAI require that [third-party AI features are enabled](group/manage.md#enable-third-party-ai-features) (which they are by default). The table above shows which features are built on which LLM. To disable AI features powered by third-party APIs, clear this setting. - -Code Suggestions currently has its own settings: - -- View [how to enable for self-managed](project/repository/code_suggestions/saas.md#enable-code-suggestions). -- View [how to enable for SaaS](project/repository/code_suggestions/self_managed.md#enable-code-suggestions-on-self-managed-gitlab). - -The use of Code Suggestions is also subject to the [Testing Terms of Use](https://about.gitlab.com/handbook/legal/testing-agreement/). - -![Settings to enable AI/ML features](img/enable_AI_ML_features.png) +- Third-party AI features + - All features built on large language models (LLM) from Google, + Anthropic or OpenAI (besides Code Suggestions) require that this setting is + enabled at the group level. + - [Generally Available](../policy/experiment-beta-support.md#generally-available-ga) + features are available when third-party AI features are enabled. + - Third-party AI features are enabled by default. + - This setting is available to Ultimate groups on SaaS and can be + set by a user who has the Owner role in the group. + - View [how to enable this setting](group/manage.md#enable-third-party-ai-features). +- Experiment and Beta features + - All features categorized as + [Experiment features](../policy/experiment-beta-support.md#experiment) or + [Beta features](../policy/experiment-beta-support.md#beta) + (besides Code Suggestions) require that this setting is enabled at the group + level. This is in addition to the Third-party AI features setting. + - Their usage is subject to the + [Testing Terms of Use](https://about.gitlab.com/handbook/legal/testing-agreement/). + - Experiment and Beta features are disabled by default. + - This setting is available to Ultimate groups on SaaS and can be set by a user + who has the Owner role in the group. + - View [how to enable this setting](group/manage.md#enable-experiment-and-beta-features). +- Code Suggestions + - View [how to enable for self-managed](project/repository/code_suggestions/self_managed.md#enable-code-suggestions-on-self-managed-gitlab). + - View [how to enable for SaaS](project/repository/code_suggestions/saas.md#enable-code-suggestions). ## Experimental AI features and how to use them @@ -51,7 +62,12 @@ The following subsections describe the experimental AI features in more detail. > Introduced in GitLab 15.11 as an [Experiment](../policy/experiment-beta-support.md#experiment) on GitLab.com. -This AI feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by Google's Codey for Code Chat (codechat-bison). +To use this feature: + +- The parent group of the project must: + - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). + - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). +- You must be a member of the project with sufficient permissions to view the repository. GitLab can help you get up to speed faster if you: @@ -60,14 +76,6 @@ GitLab can help you get up to speed faster if you: By using a large language model, GitLab can explain the code in natural language. -Prerequisites: - -Additional prerequisites in addition to [the settings listed previously](#enable-aiml-features). - -- The project must be on GitLab.com. -- You must have the GitLab Ultimate subscription tier. -- You must be a member of the project with sufficient permissions to view the repository. - To explain your code: 1. On the left sidebar, select **Search or go to** and find your project. @@ -96,13 +104,14 @@ code in a merge request: We cannot guarantee that the large language model produces results that are correct. Use the explanation with caution. -### Answer questions with Chat **(ULTIMATE SAAS EXPERIMENT)** +### Answer questions with GitLab Duo Chat **(ULTIMATE SAAS EXPERIMENT)** > Introduced in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com. It requires the [group-level third-party AI features setting](group/manage.md#enable-third-party-ai-features) to be enabled. +To use this feature, at least one group you're a member of must: -GitLab Duo Chat is powered by Anthropic's Claude-2.0 and Claude-instant-1.1 large language models and OpenAI's text-embedding-ada-002 embeddings. The LLMs are employed to analyze user questions to collect appropriate context data from the user's project, and to generate responses. In some cases, embeddings are used to embed user questions and find relevant content in GitLab documentation to share with the LLMs to generate an answer. +- Have the [third-party AI features setting](group/manage.md#enable-third-party-ai-features) enabled. +- Have the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features) enabled. You can get AI generated support from GitLab Duo Chat about the following topics: @@ -145,8 +154,12 @@ Only the last 50 messages are retained in the chat history. The chat history exp > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10344) in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by OpenAI's -GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#enable-third-party-ai-features) to be enabled. +To use this feature: + +- The parent group of the issue must: + - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). + - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). +- You must be a member of the project with sufficient permissions to view the issue. You can generate a summary of discussions on an issue: @@ -165,7 +178,12 @@ language model referenced above. > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10228) in GitLab 16.2 as an [Experiment](../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com. +To use this feature: + +- The parent group of the project must: + - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). + - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). +- You must be a member of the project with sufficient permissions to view the CI/CD analytics. In CI/CD Analytics, you can view a forecast of deployment frequency: @@ -182,12 +200,31 @@ For example, if you select a 30-day range, a forecast for the following 15 days Provide feedback on this experimental feature in [issue 416833](https://gitlab.com/gitlab-org/gitlab/-/issues/416833). +### Root cause analysis **(ULTIMATE SAAS EXPERIMENT)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123692) in GitLab 16.2 as an [Experiment](../policy/experiment-beta-support.md#experiment). + +To use this feature: + +- The parent group of the project must: + - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). + - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). +- You must be a member of the project with sufficient permissions to view the CI/CD job. + +When the feature is available, the "Root cause analysis" button will appears on +a failed CI/CD job. Selecting this button generates an analysis regarding the +reason for the failure. + ### Summarize an issue with Issue description generation **(ULTIMATE SAAS EXPERIMENT)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10762) in GitLab 16.3 as an [Experiment](../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by OpenAI's -GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#enable-third-party-ai-features) to be enabled. +To use this feature: + +- The parent group of the project must: + - Enable the [third-party AI features setting](group/manage.md#enable-third-party-ai-features). + - Enable the [experiment and beta features setting](group/manage.md#enable-experiment-and-beta-features). +- You must be a member of the project with sufficient permissions to view the issue. You can generate the description for an issue from a short summary. @@ -202,7 +239,7 @@ Provide feedback on this experimental feature in [issue 409844](https://gitlab.c **Data usage**: When you use this feature, the text you enter is sent to the large language model referenced above. -## Data Usage +## Data usage GitLab AI features leverage generative AI to help increase velocity and aim to help make you more productive. Each feature operates independently of other features and is not required for other features to function. @@ -232,5 +269,6 @@ Generative AI may produce unexpected results that may be: - Produce failed pipelines - Insecure code - Offensive or insensitive +- Out of date information GitLab is actively iterating on all our AI-assisted capabilities to improve the quality of the generated content. We improve the quality through prompt engineering, evaluating new AI/ML models to power these features, and through novel heuristics built into these features directly. diff --git a/doc/user/analytics/analytics_dashboards.md b/doc/user/analytics/analytics_dashboards.md index 68b9fef5cc7..448a46fdc26 100644 --- a/doc/user/analytics/analytics_dashboards.md +++ b/doc/user/analytics/analytics_dashboards.md @@ -67,10 +67,28 @@ This feature is not ready for production use. You can use the dashboard designer to: -- Create custom dashboards. -- Rename custom dashboards. -- Add visualizations to new and existing custom dashboards. -- Resize or move panels in custom dashboards. +- [Create custom dashboards](#create-a-custom-dashboard). +- [Edit custom dashboards](#edit-a-custom-dashboard) to: + - Rename the dashboard. + - Add and remove visualizations. + - Resize or move panels. + +## Visualization designer + +> Introduced in GitLab 16.4 [with a flag](../../administration/feature_flags.md) named `combined_analytics_visualization_editor`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `combined_analytics_visualization_editor`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +NOTE: +This feature is only compatible with the [product analytics](../product_analytics/index.md) data source. + +You can use the dashboard designer to: + +- [Create custom visualizations](#create-a-custom-visualization). +- Explore available data. ## View project dashboards @@ -201,6 +219,21 @@ To edit an existing custom dashboard: 1. Optional. In the dashboard, select a panel and drag or resize it how you prefer. 1. Select **Save**. +## Create a custom visualization + +To create a custom visualization: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Analyze > Analytics dashboards**. +1. Select **Visualization Designer**. +1. In the **Visualization title** field, enter the name of your visualization. +1. From the **Visualization type** dropdown list, select a visualization type. +1. In the **What metric do you want to visualize?** section, select the metric you want to query. +1. Optional. To refine your query, select a dimension. +1. Select **Save**. + +After you saved a visualization, you can add it to a new or existing custom dashboard in the same project. + ## Troubleshooting ### `Something went wrong while loading the dashboard.` diff --git a/doc/user/analytics/value_streams_dashboard.md b/doc/user/analytics/value_streams_dashboard.md index ed637dd886f..45be6f5aa25 100644 --- a/doc/user/analytics/value_streams_dashboard.md +++ b/doc/user/analytics/value_streams_dashboard.md @@ -119,8 +119,6 @@ To view the value streams dashboard: You can customize the Value Streams Dashboard and configure what subgroups and projects to include in the page. -A view can display maximum four subgroups or projects. - ### Using query parameters To display multiple subgroups and projects, specify their path as a URL parameter. @@ -204,6 +202,8 @@ panels: <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview of editing label filters in the configuration file, see [GitLab Value Streams Dashboard - Label filters demo](https://www.youtube.com/watch?v=4qDAHCxCfik). +Label filters are appended as query parameters to the URL of the drill-down report of each eligible metric and automatically applied. + ## Dashboard metrics and drill-down reports | Metric | Description | Drill-down report | Documentation page | ID | diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 1e9163a4c26..98b91ce584d 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -53,7 +53,7 @@ You can configure the following security controls: For more details, read [DAST on-demand scans](../dast/proxy-based.md#on-demand-scans). - [Dependency Scanning](../dependency_scanning/index.md) - Select **Configure with a merge request** to create a merge request with the changes required to - enable Dependency Scanning. For more details, see [Enable Dependency Scanning via an automatic merge request](../dependency_scanning/index.md#enable-dependency-scanning-via-an-automatic-merge-request). + enable Dependency Scanning. For more information, see [Use a preconfigured merge request](../dependency_scanning/index.md#use-a-preconfigured-merge-request). - [Container Scanning](../container_scanning/index.md) - Select **Configure with a merge request** to create a merge request with the changes required to enable Container Scanning. For more details, see diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 04b0bace265..6ee8be822da 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -189,6 +189,7 @@ variables: CS_IMAGE: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<image>:<tag> CS_REGISTRY_USER: AWS CS_REGISTRY_PASSWORD: "$AWS_ECR_PASSWORD" + AWS_DEFAULT_REGION: <region> ``` Authenticating to a remote registry is not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. @@ -783,6 +784,21 @@ To prevent the error, ensure the Docker version that the runner is using is For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). +### `unexpected status code 401 Unauthorized: Not Authorized` when scanning an image from AWS ECR + +This might happen when AWS region is not configured and the scanner cannot retrieve an authorization token. When you set `SECURE_LOG_LEVEL` to `debug` you will see a log message like below: + +```shell +[35mDEBUG[0m failed to get authorization token: MissingRegion: could not find region configuration +``` + +To resolve this, add the `AWS_DEFAULT_REGION` to your CI/CD variables: + +```yaml +variables: + AWS_DEFAULT_REGION: <AWS_REGION_FOR_ECR> +``` + ## Changes Changes to the container scanning analyzer can be found in the project's diff --git a/doc/user/application_security/dast/authentication.md b/doc/user/application_security/dast/authentication.md index aed4066bc52..c245361a731 100644 --- a/doc/user/application_security/dast/authentication.md +++ b/doc/user/application_security/dast/authentication.md @@ -63,24 +63,24 @@ To run a DAST authenticated scan: ### Available CI/CD variables -| CI/CD variable | Type | Description | -|:-----------------------------------------------|:------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `DAST_AUTH_COOKIES` | string | Set to a comma-separated list of cookie names to specify which cookies are used for authentication. | -| `DAST_AUTH_REPORT` | boolean | Set to `true` to generate a report detailing steps taken during the authentication process. You must also define `gl-dast-debug-auth-report.html` as a CI job artifact to be able to access the generated report. The report's content aids when debugging authentication failures. | -| `DAST_AUTH_TYPE` <sup>2</sup> | string | The authentication type to use. Example: `basic-digest`. | -| `DAST_AUTH_URL` <sup>1</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Example: `https://login.example.com`. | -| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the absence of a login form after the login form has been submitted. | -| `DAST_AUTH_VERIFICATION_SELECTOR` | [selector](#finding-an-elements-selector) | Verifies successful authentication by checking for presence of a selector after the login form has been submitted. Example: `css:.user-photo`. | -| `DAST_AUTH_VERIFICATION_URL` <sup>1</sup> | URL | Verifies successful authentication by checking the URL in the browser after the login form has been submitted. Example: `"https://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | -| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1</sup> | [selector](#finding-an-elements-selector) | Comma-separated list of selectors that are selected prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | -| `DAST_EXCLUDE_URLS` <sup>1</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | -| `DAST_FIRST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | -| `DAST_PASSWORD` <sup>1</sup> | string | The password to authenticate to in the website. Example: `P@55w0rd!` | -| `DAST_PASSWORD_FIELD` | string | The selector of password field at the sign-in HTML form. Example: `id:password` | -| `DAST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | -| `DAST_USERNAME` <sup>1</sup> | string | The username to authenticate to in the website. Example: `admin` | -| `DAST_USERNAME_FIELD` <sup>1</sup> | string | The selector of username field at the sign-in HTML form. Example: `name:username` | -| `DAST_AUTH_DISABLE_CLEAR_FIELDS` | boolean | Disables clearing of username and password fields before attempting manual login. Set to `false` by default. | +| CI/CD variable | Type | Description | +|:-----------------------------------------------|:------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `DAST_AUTH_COOKIES` | string | Set to a comma-separated list of cookie names to specify which cookies are used for authentication. | +| `DAST_AUTH_REPORT` | boolean | Set to `true` to generate a report detailing steps taken during the authentication process. You must also define `gl-dast-debug-auth-report.html` as a CI job artifact to be able to access the generated report. The report's content aids when debugging authentication failures. | +| `DAST_AUTH_TYPE` <sup>2</sup> | string | The authentication type to use. Example: `basic-digest`. | +| `DAST_AUTH_URL` <sup>1</sup> | URL | The URL of the page containing the login form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Example: `https://login.example.com`. | +| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the absence of a login form after the login form has been submitted. | +| `DAST_AUTH_VERIFICATION_SELECTOR` | [selector](#finding-an-elements-selector) | A selector describing an element whose presence is used to determine if authentication has succeeded after the login form is submitted. Example: `css:.user-photo`. | +| `DAST_AUTH_VERIFICATION_URL` <sup>1</sup> | URL | A URL that is compared to the URL in the browser to determine if authentication has succeeded after the login form is submitted. Example: `"https://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | +| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1</sup> | [selector](#finding-an-elements-selector) | A comma-separated list of selectors representing elements to click on prior to entering the `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | +| `DAST_EXCLUDE_URLS` <sup>1</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | +| `DAST_FIRST_SUBMIT_FIELD` | [selector](#finding-an-elements-selector) | A selector describing the element that is clicked on to submit the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | +| `DAST_PASSWORD` <sup>1</sup> | string | The password to authenticate to in the website. Example: `P@55w0rd!` | +| `DAST_PASSWORD_FIELD` | [selector](#finding-an-elements-selector) | A selector describing the element used to enter the password on the login form. Example: `id:password` | +| `DAST_SUBMIT_FIELD` | [selector](#finding-an-elements-selector) | A selector describing the element clicked on to submit the login form for a single-page login form, or the password form for a multi-page login form. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | +| `DAST_USERNAME` <sup>1</sup> | string | The username to authenticate to in the website. Example: `admin` | +| `DAST_USERNAME_FIELD` <sup>1</sup> | [selector](#finding-an-elements-selector) | A selector describing the element used to enter the username on the login form. Example: `name:username` | +| `DAST_AUTH_DISABLE_CLEAR_FIELDS` | boolean | Disables clearing of username and password fields before attempting manual login. Set to `false` by default. | 1. Available to an on-demand proxy-based DAST scan. 1. Not available to proxy-based scans. diff --git a/doc/user/application_security/dast/checks/611.1.md b/doc/user/application_security/dast/checks/611.1.md new file mode 100644 index 00000000000..49ef449f8b0 --- /dev/null +++ b/doc/user/application_security/dast/checks/611.1.md @@ -0,0 +1,31 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# External XML Entity Injection (XXE) + +## Description + +It is possible to cause the application's XML parser to include external resources. +This can include files or in some circumstances initiate requests to third party +servers. + +## Remediation + +Consult the documentation for the XML Parser used by the target application for security +guidelines and hardening steps. It is recommended that all XML parsers disable external +entity resolution and XML `xinclude` features. Most XML parsers based on `libxml` can also be +configured to disable network access. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 611.1 | false | 611 | Active | high | + +## Links + +- [OWASP](https://owasp.org/www-community/vulnerabilities/XML_External_Entity_(XXE)_Processing) +- [CWE](https://cwe.mitre.org/data/definitions/611.html) diff --git a/doc/user/application_security/dast/checks/94.4.md b/doc/user/application_security/dast/checks/94.4.md new file mode 100644 index 00000000000..9dddada84f9 --- /dev/null +++ b/doc/user/application_security/dast/checks/94.4.md @@ -0,0 +1,49 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Server-side code injection (NodeJS) + +## Description + +The target application was found vulnerable to code injection. A malicious actor could inject arbitrary +JavaScript code to be executed on the server. This could lead to a full system compromise by accessing +stored secrets, injecting code to take over accounts, or executing OS commands. + +## Remediation + +Never pass user input directly into functions which evaluate string data as code, such as `eval`, `setTimeout` +or `setInterval`. There is almost no benefit of passing string values to these methods, as such the best +recommendation is to replace the current logic with more safe implementations of dynamically evaluating +logic with user input. One alternative is to store functions or methods in a Map that can be looked +up using a key. If the key exists, the function can be executed. + +```javascript +const function_map = new Map(); + +function_map.set('fn', function() { + console.log('hello world'); +}) + +const input = 'fn2'; + +const fn = function_map.get(input) + +if (fn) { + fn(); +} else { + console.log('invalid input'); +} +``` + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 94.4 | false | 94 | Active | high | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/94.html) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index 035f4a4b486..4d41f08672e 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -169,3 +169,5 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne |:---|:------|:---------|:-----| | [113.1](113.1.md) | Improper Neutralization of CRLF Sequences in HTTP Headers | High | Active | | [22.1](22.1.md) | Improper limitation of a pathname to a restricted directory (Path traversal) | High | Active | +| [611.1](611.1.md) | External XML Entity Injection (XXE) | High | Active | +| [94.4](94.4.md) | Server-side code injection (NodeJS) | High | Active | diff --git a/doc/user/application_security/dast/proxy-based.md b/doc/user/application_security/dast/proxy-based.md index 86af7d4c5da..230d8ef5ca3 100644 --- a/doc/user/application_security/dast/proxy-based.md +++ b/doc/user/application_security/dast/proxy-based.md @@ -464,7 +464,6 @@ The DAST job does not require the project's repository to be present when runnin > - Auditing for DAST profile management [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. > - Scheduled on-demand DAST scans [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.3 [with a flag](../../../administration/feature_flags.md) named `dast_on_demand_scans_scheduler`. Disabled by default. > - Scheduled on-demand DAST scans [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.5. Feature flag `dast_on_demand_scans_scheduler` removed. -> - Runner tags selection [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345430) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `on_demand_scans_runner_tags. Disabled by default. > - Runner tags selection [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111499) in GitLab 16.3. WARNING: diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index d8726cbd456..91145b10f81 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -10,6 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - System dependencies [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6698) in GitLab 14.6. > - Group-level dependency list [introduced](https://gitlab.com/groups/gitlab-org/-/epics/8090) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `group_level_dependencies`. Disabled by default. > - Group-level dependency list [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/411257) in GitLab 16.4. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/132015) in GitLab 16.5. Feature flag `group_level_dependencies` removed. Use the dependency list to review your project or group's dependencies and key details about those dependencies, including their known vulnerabilities. This list is a collection of dependencies in your diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index f8bd5b99cb6..c04134de2b2 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -6,6 +6,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Dependency Scanning **(ULTIMATE ALL)** +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo of this Dependency Scanning doc, see [How to use dependency scanning tutorial hands-on GitLab Application Security part 3](https://youtu.be/ii05cMbJ4xQ?feature=shared) +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) + Dependency Scanning analyzes your application's dependencies for known vulnerabilities. All dependencies are scanned, including transitive dependencies, also known as nested dependencies. @@ -31,17 +36,6 @@ we encourage you to use all of our security scanners. For a comparison of these <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o). -## Requirements - -Dependency Scanning runs in the `test` stage, which is available by default. If you redefine the -stages in the `.gitlab-ci.yml` file, the `test` stage is required. - -To run dependency scanning jobs, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared runners on GitLab.com, this is enabled by default. The analyzer images -provided are for the Linux/amd64 architecture. - WARNING: Dependency Scanning does not support runtime installation of compilers and interpreters. @@ -126,7 +120,7 @@ table.supported-languages ul { 8 LTS, 11 LTS, 17 LTS, - or 21 EA<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-2">2</a></b></sup> + or 21 LTS<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-2">2</a></b></sup> </td> <td><a href="https://gradle.org/">Gradle</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-3">3</a></b></sup></td> <td> @@ -236,8 +230,7 @@ table.supported-languages ul { <li> <a id="notes-regarding-supported-languages-and-package-managers-2"></a> <p> - Java 21 EA is only available when using <a href="https://maven.apache.org/">Maven</a> and is not supported when - <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. + Java 21 LTS is only available when using <a href="https://maven.apache.org/">Maven</a> or <a href="https://gradle.org/">Gradle</a>. Java 21 LTS for <a href="https://www.scala-sbt.org/">sbt</a> is not yet available and tracked in <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/421174">issue 421174</a>. It is not supported when <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. </p> </li> <li> @@ -422,7 +415,7 @@ To support the following package managers, the GitLab analyzers proceed in two s <p> If your project <i>does not use</i> a <code>gradlew</code> file, then the analyzer automatically switches to one of the pre-installed Gradle versions, based on the version of Java specified by the - <a href="#configuring-specific-analyzers-used-by-dependency-scanning"><code>DS_JAVA_VERSION</code></a> variable. + <a href="#analyzer-specific-settings"><code>DS_JAVA_VERSION</code></a> variable. By default, the analyzer uses Java 17 and Gradle 7.3.3. </p> <p> @@ -532,58 +525,88 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Configuration -To enable dependency scanning for GitLab 11.9 and later, you must -[include](../../../ci/yaml/index.md#includetemplate) the -[`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) -that is provided as a part of your GitLab installation. -For GitLab versions earlier than 11.9, you can copy and use the job as defined -that template. +Enable the dependency scanning analyzer to ensure it scans your application's dependencies for known +vulnerabilities. You can then adjust its behavior by using CI/CD variables. -Add the following to your `.gitlab-ci.yml` file: +### Enabling the analyzer -```yaml -include: - - template: Jobs/Dependency-Scanning.gitlab-ci.yml -``` +Prerequisites: + +- The `test` stage is required in the `.gitlab-ci.yml` file. +- On GitLab self-managed you need GitLab Runner with the + [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or + [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. On GitLab.com this + is enabled by default on the shared runners. The analyzer images provided are for the Linux/amd64 + architecture. -The included template creates dependency scanning jobs in your CI/CD -pipeline and scans your project's source code for possible vulnerabilities. -The results are saved as a -[dependency scanning report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsdependency_scanning) -that you can later download and analyze. Due to implementation limitations, we -always take the latest dependency scanning artifact available. +To enable the analyzer, either: -### Enable Dependency Scanning via an automatic merge request +- Enable [Auto DevOps](../../../topics/autodevops/index.md), which includes dependency scanning. +- Edit the `.gitlab-ci.yml` file manually. Use this method if your `.gitlab-ci.yml` file is complex. +- Use a preconfigured merge request. +- Create a [scan execution policy](../policies/scan-execution-policies.md) that enforces dependency + scanning. + +#### Edit the `.gitlab-ci.yml` file manually + +This method requires you to manually edit the existing `.gitlab-ci.yml` file. Use this method if +your GitLab CI/CD configuration file is complex. + +To enable dependency scanning: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipeline editor**. +1. Copy and paste the following to the bottom of the `.gitlab-ci.yml` file: + + ```yaml + include: + - template: Security/Dependency-Scanning.gitlab-ci.yml + ``` + +1. Select the **Validate** tab, then select **Validate pipeline**. + + Continue if you see the message **Simulation completed successfully**. That indicates the file is + valid. +1. Select the **Edit** tab. +1. Complete the fields. Do not use the default branch for the **Branch** field. +1. Select **Commit changes**. +1. Select **Code > Merge requests**. +1. Select the merge request just created. +1. Review the merge request, then select **Merge**. + +Pipelines now include a dependency scanning job. + +#### Use a preconfigured merge request > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4908) in GitLab 14.1 [with a flag](../../../administration/feature_flags.md) named `sec_dependency_scanning_ui_enable`. Enabled by default. -> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/282533) in GitLab 14.1. -> - [Feature flag `sec_dependency_scanning_ui_enable` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. Feature flag `sec_dependency_scanning_ui_enable` removed. + +This method automatically prepares a merge request that includes the dependency scanning template +in the `.gitlab-ci.yml` file. You then merge the merge request to enable dependency scanning. -To enable Dependency Scanning in a project, you can create a merge request: +NOTE: +This method works best with no existing `.gitlab-ci.yml` file, or with a minimal configuration +file. If you have a complex GitLab configuration file it might not be parsed successfully, and an +error might occur. In that case, use the [manual](#edit-the-gitlab-ciyml-file-manually) method instead. + +To enable dependency scanning: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Secure > Security configuration**. 1. In the **Dependency Scanning** row, select **Configure with a merge request**. -1. Review and merge the merge request to enable Dependency Scanning. +1. Select **Create merge request**. +1. Review the merge request, then select **Merge**. Pipelines now include a dependency scanning job. -### Customizing the dependency scanning settings - -The Dependency Scanning settings can be changed through [CI/CD variables](#available-cicd-variables) by using the -[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. -For example: +### Customizing analyzer behavior -```yaml -include: - - template: Security/Dependency-Scanning.gitlab-ci.yml +You can use CI/CD variables to customize dependency scanning behavior. -variables: - SECURE_LOG_LEVEL: error -``` - -Because template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline -configuration, the last mention of the variable takes precedence. +WARNING: +You should test all customization of GitLab security scanning tools in a merge request before +merging these changes to the default branch. Failure to do so can give unexpected results, +including a large number of false positives. ### Overriding dependency scanning jobs @@ -613,15 +636,9 @@ gemnasium-dependency_scanning: ### Available CI/CD variables -Dependency scanning can be [configured](#customizing-the-dependency-scanning-settings) -using environment variables. - -WARNING: -All customization of GitLab security scanning tools should be tested in a merge request before -merging these changes to the default branch. Failure to do so can give unexpected results, -including a large number of false positives. +You can use CI/CD variables to [customize](#customizing-analyzer-behavior) dependency scanning behavior. -#### Configuring dependency scanning +#### Global analyzer settings The following variables allow configuration of global dependency scanning settings. @@ -634,9 +651,9 @@ The following variables allow configuration of global dependency scanning settin | `DS_MAX_DEPTH` | Defines how many directory levels deep that the analyzer should search for supported files to scan. A value of `-1` scans all directories regardless of depth. Default: `2`. | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). | -#### Configuring specific analyzers used by dependency scanning +#### Analyzer-specific settings -The following variables are used for configuring specific analyzers (used for a specific language/framework). +The following variables configure the behavior of specific dependency scanning analyzers. | CI/CD variable | Analyzer | Default | Description | |--------------------------------------| ------------------ | ---------------------------- |------------ | @@ -957,10 +974,10 @@ jobs to run successfully. For more information, see [Offline environments](../of Here are the requirements for using dependency scanning in an offline environment: -- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- GitLab Runner with the `docker` or `kubernetes` executor. - Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - If you have a limited access environment you need to allow access, such as using a proxy, to the advisory database: `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git`. - If you are unable to permit access to `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` you must host an offline copy of this `git` repository and set the `GEMNASIUM_DB_REMOTE_URL` CI/CD variable to the URL of this repository. For more information on configuration variables, see [Dependency Scanning](#configuring-dependency-scanning). + If you are unable to permit access to `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` you must host an offline copy of this `git` repository and set the `GEMNASIUM_DB_REMOTE_URL` CI/CD variable to the URL of this repository. For more information on configuration variables, see [Customizing analyzer behavior](#customizing-analyzer-behavior). This advisory database is constantly being updated, so you must periodically sync your local copy with GitLab. @@ -1019,7 +1036,7 @@ variables: GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" ``` -See explanations of the variables above in the [configuration section](#configuration). +See explanations of the previous variables in the [configuration section](#customizing-analyzer-behavior). ### Hosting a copy of the `gemnasium_db` advisory database @@ -1353,7 +1370,7 @@ The `go.sum` file contains an entry of every module that was considered while ge Multiple versions of a module are included in the `go.sum` file, but the [MVS](https://go.dev/ref/mod#minimal-version-selection) algorithm used by `go build` only selects one. As a result, when dependency scanning uses `go.sum`, it might report false positives. -To prevent false positives, gemnasium only uses `go.sum` if it is unable to generate the build list for the Go project. If `go.sum` is selected, a warning occurs: +To prevent false positives, Gemnasium only uses `go.sum` if it is unable to generate the build list for the Go project. If `go.sum` is selected, a warning occurs: ```shell [WARN] [Gemnasium] [2022-09-14T20:59:38Z] ▶ Selecting "go.sum" parser for "/test-projects/gitlab-shell/go.sum". False positives may occur. See https://gitlab.com/gitlab-org/gitlab/-/issues/321081. diff --git a/doc/user/application_security/get-started-security.md b/doc/user/application_security/get-started-security.md index ad49f00a1bd..3e73fbc5955 100644 --- a/doc/user/application_security/get-started-security.md +++ b/doc/user/application_security/get-started-security.md @@ -8,6 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Adopting GitLab application security](https://www.youtube.com/watch?v=5QlxkiKR04k). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) The following steps help you get the most from GitLab application security tools. These steps are a recommended order of operations. You can choose to implement capabilities in a different order or omit features that do not apply to your specific needs. diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 8cdeeb92e09..bca29420192 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -4,144 +4,106 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Infrastructure as Code (IaC) Scanning **(FREE ALL)** +# Infrastructure as Code scanning **(FREE ALL)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6655) in GitLab 14.5. -Infrastructure as Code (IaC) Scanning scans your IaC configuration files for known vulnerabilities. +Infrastructure as Code (IaC) scanning runs in your CI/CD pipeline, checking your infrastructure +definition files for known vulnerabilities. Identify vulnerabilities before they're committed to +the default branch to proactively address the risk to your application. -IaC Scanning supports configuration files for Terraform, Ansible, AWS CloudFormation, and Kubernetes. +The IaC scanning analyzer outputs JSON-formatted reports as +[job artifacts](../../../ci/yaml/artifacts_reports.md#artifactsreportssast). -## Requirements +With GitLab Ultimate, IaC scanning results are also processed so you can: -IaC Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the `.gitlab-ci.yml` file, the `test` stage is required. +- See them in merge requests. +- Use them in approval workflows. +- Review them in the vulnerability report. -We recommend a minimum of 4 GB RAM to ensure consistent performance. +## Enable the scanner -To run IaC Scanning jobs, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared runners on GitLab.com, this is enabled by default. +Prerequisites: -WARNING: -GitLab IaC Scanning analyzers don't support running on Windows or on any CPU architectures other than amd64. +- IaC scanning requires the AMD64 architecture. Microsoft Windows is not supported. +- Minimum of 4 GB RAM to ensure consistent performance. +- The `test` stage is required in the `.gitlab-ci.yml` file. +- On GitLab self-managed you need GitLab Runner with the + [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or + [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. On GitLab.com this + is enabled by default on the shared runners. The analyzer images provided are for the Linux/amd64 + architecture. -WARNING: -If you use your own runners, make sure the Docker version installed -is **not** `19.03.0`. See [troubleshooting information](../sast/troubleshooting.md#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. +To enable IaC scanning of a project: -## Supported languages and frameworks - -GitLab IaC Scanning supports a variety of IaC configuration files. Our IaC security scanners also feature automatic language detection which works even for mixed-language projects. If any supported configuration files are detected in project source code we automatically run the appropriate IaC analyzers. - -| Configuration file type | Scan tool | Introduced in GitLab version | -| ----------------------------------- | ------------------------ | ---------------------------- | -| Ansible | [KICS](https://kics.io/) | 14.5 | -| AWS CloudFormation | [KICS](https://kics.io/) | 14.5 | -| Azure Resource Manager <sup>1</sup> | [KICS](https://kics.io/) | 14.5 | -| Dockerfile | [KICS](https://kics.io/) | 14.5 | -| Google Deployment Manager | [KICS](https://kics.io/) | 14.5 | -| Kubernetes | [KICS](https://kics.io/) | 14.5 | -| OpenAPI | [KICS](https://kics.io/) | 14.5 | -| Terraform <sup>2</sup> | [KICS](https://kics.io/) | 14.5 | - -1. IaC Scanning can analyze Azure Resource Manager templates in JSON format. If you write templates in the [Bicep](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview) language, you must use [the bicep CLI](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-cli) to convert your Bicep files into JSON before GitLab IaC Scanning can analyze them. -1. Terraform modules in a custom registry are not scanned for vulnerabilities. You can follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/357004) for the proposed feature. - -### Supported distributions - -GitLab scanners are provided with a base alpine image for size and maintainability. - -#### FIPS-enabled images - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6479) in GitLab 14.10. - -GitLab also offers [FIPS-enabled Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) -versions of the images. You can therefore replace standard images with FIPS-enabled -images. To configure the images, set the `SAST_IMAGE_SUFFIX` to `-fips` or modify the -standard tag plus the `-fips` extension. - -```yaml -variables: - SAST_IMAGE_SUFFIX: '-fips' - -include: - - template: Jobs/SAST-IaC.gitlab-ci.yml -``` - -### Making IaC analyzers available to all GitLab tiers - -All open source (OSS) analyzers are available with the GitLab Free tier. Future proprietary analyzers may be restricted to higher tiers. - -#### Summary of features per tier - -Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), -as shown in the following table: - -| Capability | In Free & Premium | In Ultimate | -| :-------------------------------------------------------------- | :------------------ | :----------------- | -| [Configure IaC scanner](#configuration) | **{check-circle}** | **{check-circle}** | -| Download [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | -| See new findings in merge request widget | **{dotted-circle}** | **{check-circle}** | -| [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | - -## Contribute your scanner - -The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate other security scanners into GitLab. - -## Configuration - -To configure IaC Scanning for a project you can: +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Build > Pipeline editor**. +1. Copy and paste the following to the bottom of the `.gitlab-ci.yml` file. -- [Configure IaC Scanning manually](#configure-iac-scanning-manually) -- [Enable IaC Scanning via an automatic merge request](#enable-iac-scanning-via-an-automatic-merge-request) + ```yaml + include: + - template: Security/SAST-IaC.gitlab-ci.yml + ``` -### Configure IaC Scanning manually +1. Select the **Validate** tab, then select **Validate pipeline**. + The message **Simulation completed successfully** indicates the file is valid. +1. Select the **Edit** tab. +1. Select **Commit changes**. -To enable IaC Scanning you must [include](../../../ci/yaml/index.md#includetemplate) the -[`SAST-IaC.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) provided as part of your GitLab installation. Here is an example of how to include it: +Pipelines now include an IaC scanning job. -```yaml -include: - - template: Jobs/SAST-IaC.gitlab-ci.yml -``` +## Supported languages and frameworks -The included template creates IaC Scanning jobs in your CI/CD pipeline and scans -your project's configuration files for possible vulnerabilities. +IaC scanning supports a variety of IaC configuration files. When any supported configuration files +are detected in a project, they are scanned by using [KICS](https://kics.io/). Projects with a mix +of IaC configuration files are supported. -The results are saved as a -[SAST report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportssast) -that you can download and analyze. +Supported configuration formats: -### Enable IaC Scanning via an automatic merge request +- Ansible +- AWS CloudFormation +- Azure Resource Manager <sup>1</sup> +- Dockerfile +- Google Deployment Manager +- Kubernetes +- OpenAPI +- Terraform <sup>2</sup> -NOTE: -The **Configure with a merge request** button been temporarily disabled due to a known issue. For details, see [merge request 83757](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83757). +<html> +<small>Footnotes: + <ol> + <li>IaC Scanning can analyze Azure Resource Manager templates in JSON format. If you write templates in the <a href="https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview">Bicep</a> language, you must use the <a href="https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-cli">bicep CLI</a> to convert your Bicep files into JSON before IaC scanning can analyze them.</li> + <li>Terraform modules in a custom registry are not scanned for vulnerabilities. You can follow <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/357004">issue 357004</a> for the proposed feature.</li> + </ol> +</small> +</html> -To enable IaC Scanning in a project, you can create a merge request: +## Customize rules **(ULTIMATE ALL)** -1. On the left sidebar, select **Search or go to** and find your project. -1. Select **Secure > Security configuration**. -1. In the **Infrastructure as Code (IaC) Scanning** row, select **Configure with a merge request**. -1. Review and merge the merge request to enable IaC Scanning. +> Support for overriding rules [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) in GitLab 14.8. -Pipelines now include an IaC Scanning job. +You can customize the default IaC scanning rules provided with GitLab. -## Customize rulesets **(ULTIMATE ALL)** +The following customization options can be used separately, or together: -> Support for overriding rules [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) in GitLab 14.8. +- [Disable predefined rules](#disable-rules). +- [Override predefined rules](#override-rules). -You can customize the default IaC Scanning rules provided with GitLab. +### Ruleset definition -The following customization options can be used separately, or together: +Every IaC scanning rule is contained in a `ruleset` section, which contains: -- [Disable predefined rules](#disable-predefined-analyzer-rules). -- [Override predefined rules](#override-predefined-analyzer-rules). +- A `type` field for the rule. For IaC Scanning, the identifier type is `kics_id`. +- A `value` field for the rule identifier. KICS rule identifiers are alphanumeric strings. + To find the rule identifier: + - Find it in the [JSON report artifact](#reports-json-format). + - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) + and copy the alphanumeric identifier that's shown. The rule name is shown on the + [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected. -### Disable predefined analyzer rules +### Disable rules -If there are specific IaC Scanning rules that you don't want active, you can disable them. +You can disable specific IaC Scanning rules. To disable analyzer rules: @@ -149,12 +111,7 @@ To disable analyzer rules: 1. Create a custom ruleset file named `sast-ruleset.toml` in the `.gitlab` directory, if one doesn't already exist. 1. Set the `disabled` flag to `true` in the context of a `ruleset` section. -1. In one or more `ruleset` subsections, list the rules to disable. Every - `ruleset.identifier` section has: - - A `type` field for the rule. For IaC Scanning, the identifier type is `kics_id`. - - A `value` field for the rule identifier. KICS rule identifiers are alphanumeric strings. To find the rule identifier, you can: - - Find it in the [JSON report artifact](#reports-json-format). - - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) and copy the alphanumeric identifier that's shown. The rule name is shown on the [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected. +1. In one or more `ruleset` subsections, list the rules to disable. After you merge the `sast-ruleset.toml` file to the default branch, existing findings for disabled rules are [automatically resolved](#automatic-vulnerability-resolution). @@ -176,25 +133,19 @@ the `kics` analyzer by matching the `type` and `value` of identifiers: value = "b03a748a-542d-44f4-bb86-9199ab4fd2d5" ``` -### Override predefined analyzer rules +### Override rules -If there are specific IaC Scanning rules you want to customize, you can override them. For -example, you might lower the severity of a rule or link to your own documentation about how to fix a finding. +You can override specific IaC scanning rules to customize them. For example, assign a rule a lower +severity, or link to your own documentation about how to fix a finding. To override rules: 1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. 1. Create a custom ruleset file named `sast-ruleset.toml` in the `.gitlab` directory, if one doesn't already exist. -1. In one or more `ruleset.identifier` subsections, list the rules to override. Every - `ruleset.identifier` section has: - - A `type` field for the rule. For IaC Scanning, the identifier type is `kics_id`. - - A `value` field for the rule identifier. KICS rule identifiers are alphanumeric strings. To find the rule identifier, you can: - - Find it in the [JSON report artifact](#reports-json-format). - - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) and copy the alphanumeric identifier that's shown. The rule name is shown on the [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected. -1. In the `ruleset.override` context of a `ruleset` section, - provide the keys to override. Any combination of keys can be - overridden. Valid keys are: +1. In one or more `ruleset.identifier` subsections, list the rules to override. +1. In the `ruleset.override` context of a `ruleset` section, provide the keys to override. Any + combination of keys can be overridden. Valid keys are: - description - message - name @@ -216,26 +167,30 @@ In the following example `sast-ruleset.toml` file, rules are matched by the `typ severity = "Info" ``` -## Pinning to specific analyzer version +## Use a specific analyzer version -The GitLab-managed CI/CD template specifies a major version and automatically pulls the latest analyzer release within that major version. - -In some cases, you may need to use a specific version. +The GitLab-managed CI/CD template specifies a major version and automatically pulls the latest +analyzer release in that major version. In some cases, you may need to use a specific version. For example, you might need to avoid a regression in a later release. -To override the automatic update behavior, set the `SAST_ANALYZER_IMAGE_TAG` CI/CD variable -in your CI/CD configuration file after you include the [`SAST-IaC.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml). +To use a specific analyzer version: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Build > Pipeline editor**. +1. Add the `SAST_ANALYZER_IMAGE_TAG` CI/CD variable, after the line that includes the + `SAST-IaC.gitlab-ci.yml` template. -Only set this variable in a specific job. -If you set it [at the top level](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file), the version you set is used for other SAST analyzers. + NOTE: + Only set this variable in a specific job. If you set it at the top level, the version you set is + used for other SAST analyzers. -You can set the tag to: + Set the tag to: -- A major version, like `3`. Your pipelines use any minor or patch updates that are released within this major version. -- A minor version, like `3.7`. Your pipelines use any patch updates that are released within this minor version. -- A patch version, like `3.7.0`. Your pipelines don't receive any updates. + - A major version, like `3`. Your pipelines use any minor or patch updates that are released in this major version. + - A minor version, like `3.7`. Your pipelines use any patch updates that are released in this minor version. + - A patch version, like `3.7.0`. Your pipelines don't receive any updates. -This example uses a specific minor version of the `KICS` analyzer: +This example uses a specific minor version of the IaC analyzer: ```yaml include: @@ -246,46 +201,73 @@ kics-iac-sast: SAST_ANALYZER_IMAGE_TAG: "3.1" ``` +## Supported distributions + +GitLab scanners are provided with a base Alpine image for size and maintainability. + +### Use FIPS-enabled images + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6479) in GitLab 14.10. + +GitLab provides [FIPS-enabled Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) +versions of the scanners' images, in addition to the standard images. + +To use the FIPS-enabled images in a pipeline, set the `SAST_IMAGE_SUFFIX` to `-fips` or modify the +standard tag plus the `-fips` extension. + +The following example uses the `SAST_IMAGE_SUFFIX` CI/CD variable. + +```yaml +variables: + SAST_IMAGE_SUFFIX: '-fips' + +include: + - template: Security/SAST-IaC.gitlab-ci.yml +``` + ## Automatic vulnerability resolution > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. -> - Enabled by default in 15.10. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/375128) in GitLab 16.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/375128) in GitLab 16.2. Feature flag `sec_mark_dropped_findings_as_resolved` removed. -To help you focus on the vulnerabilities that are still relevant, GitLab IaC Scanning automatically [resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: +To help you focus on the vulnerabilities that are still relevant, IaC scanning automatically +[resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: -- You [disable a predefined rule](#disable-predefined-analyzer-rules). +- You [disable a predefined rule](#disable-rules). - We remove a rule from the default ruleset. -The Vulnerability Management system leaves a comment on automatically-resolved vulnerabilities so you still have a historical record of the vulnerability. - If you re-enable the rule later, the findings are reopened for triage. +The vulnerability management system adds a note when it automatically resolves a vulnerability. + ## Reports JSON format -The IaC tool emits a JSON report file in the existing SAST report format. For more information, see the +The IaC scanner outputs a JSON report file in the existing SAST report format. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). -The JSON report file can be downloaded from the CI pipelines page, or the -pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/jobs/job_artifacts.md). +The JSON report file can be downloaded from: -## Troubleshooting +- The CI pipelines page. +- The pipelines tab on merge requests by + [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. + +For more information see [Downloading artifacts](../../../ci/jobs/job_artifacts.md). -### Debug-level logging +## Troubleshooting -Debug-level logging can help when troubleshooting. For details, see -[debug-level logging](../index.md#debug-level-logging). +When working with IaC scanning, you might encounter the following issues. -### IaC Scanning findings show as `No longer detected` unexpectedly +### IaC scanning findings show as `No longer detected` unexpectedly If a previously detected finding unexpectedly shows as `No longer detected`, it might be due to an update to the scanner. An update can disable rules that are found to -be ineffective or false positives, and the findings are marked as `No longer detected`: +be ineffective or false positives, and the findings are marked as `No longer detected`. -- In GitLab 15.3, [secret detection in the KICS SAST IaC scanner was disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/346181), - so IaC findings in the "Passwords and Secrets" family show as `No longer detected`. +In GitLab 15.3, [secret detection in the IaC scanner was disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/346181), +so IaC findings in the "Passwords and Secrets" family show as `No longer detected`. -### `exec /bin/sh: exec format error` message in job log +### Message `exec /bin/sh: exec format error` in job log -The GitLab IaC Scanning analyzer [only supports](#requirements) running on the `amd64` CPU architecture. -This message indicates that the job is being run on a different architecture, such as `arm`. +You might get an error in the job log that states `exec /bin/sh: exec format error`. This issue +occurs when attempting to run the IaC scanning analyzer on an architecture other than AMD64 +architecture. For details of IaC scanning prerequisites, see [Enable the scanner](#enable-the-scanner). diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 84c28d4008c..a86d9b63c63 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -143,15 +143,14 @@ The workaround is to amend your group or instance push rules to allow branches f - Confirm that scanners are properly configured and producing results for the latest branch. Security Policies are designed to require approval when there are no results (no security report), as this ensures that no vulnerabilities are introduced. We cannot know if there are any vulnerabilities unless the scans enforced by the policy complete successfully and are evaluated. - For scan result policies, we require artifacts for each scanner defined in the policy for both the source and target branch. To ensure scan result policies capture the necessary results, confirm your scan execution is properly implemented and enforced. If using scan execution policies, enforcing on `all branches` often address this need. -- When running scan execution policies based on a SAST action, ensure target repositories contain proper code files. SAST runs different analyzers [based on the types of files in the repo](../sast/index.md#supported-languages-and-frameworks), and if no supported files are found it will not run any jobs. See the [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) for more details. +- When running scan execution policies based on a SAST action, ensure target repositories contain proper code files. SAST runs different analyzers [based on the types of files in the repository](../sast/index.md#supported-languages-and-frameworks), and if no supported files are found it will not run any jobs. See the [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) for more details. - Check for any branch configuration conflicts. If your policy is configured to enforce rules on `main` but some projects within the scope are using `master` as their default branch, the policy is not applied for the latter. You can define policies to enforce rules generically on `default` branches regardless of the name used in the project or on `all protected branches` to address this issue. - Scan result policies created at the group or sub-group level can take some time to apply to all the merge requests in the group. - Scheduled scan execution policies run with a minimum 15 minute cadence. Learn more [about the schedule rule type](../policies/scan-execution-policies.md#schedule-rule-type). - When scheduling pipelines, keep in mind that CRON scheduling is based on UTC on GitLab SaaS and is based on your server time for self managed instances. When testing new policies, it may appear pipelines are not running properly when in fact they are scheduled in your server's timezone. -- When enforcing scan execution policies, security policies creates a bot in the target project that will trigger scheduled pipelines to ensure enforcement. If the bot is -deleted or missing, the target project's pipeline will not be executed. To recreate a security policy bot user unlink and link the security policy project again. +- When enforcing scan execution policies, security policies use a bot in the target project that will trigger scheduled pipelines to ensure enforcement. When the bot is missing, it will be automatically created, and the following scheduled scan will use it. - You should not link a security policy project to a development project and to the group or sub-group the development project belongs to at the same time. Linking this way will result in approval rules from the Scan Result Policy not being applied to merge requests in the development project. -- When creating a Scan Result Policy, neither the array `severity_levels` nor the array `vulnerability_states` in the [scan_finding rule](../policies/scan-result-policies.md#scan_finding-rule-type) can be left empty; for a working rule, at least one entry must exist. +- When creating a Scan Result Policy, neither the array `severity_levels` nor the array `vulnerability_states` in the [`scan_finding` rule](../policies/scan-result-policies.md#scan_finding-rule-type) can be left empty; for a working rule, at least one entry must exist. - When configuring pipeline and scan result policies, it's important to remember that security scans performed in manual jobs aren't verified to determine whether MR approval is required. When you run a manual job with security scans, it won't ensure approval even if vulnerabilities are introduced. If you are still experiencing issues, you can [view recent reported bugs](https://gitlab.com/gitlab-org/gitlab/-/issues/?sort=popularity&state=opened&label_name%5B%5D=group%3A%3Asecurity%20policies&label_name%5B%5D=type%3A%3Abug&first_page_size=20) and raise new unreported issues. diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index ac15dfc0a47..0eb2355beb7 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Group-level security policies [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/356258) in GitLab 15.4. > - Operational container scanning [introduced](https://gitlab.com/groups/gitlab-org/-/epics/3410) in GitLab 15.5 > - Support for custom CI variables in the Scan Execution Policies editor [introduced](https://gitlab.com/groups/gitlab-org/-/epics/9566) in GitLab 16.2. -> - Enforcement of scan execution policies on projects with an existing GitLab CI/CD configuration [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6880) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `scan_execution_policy_pipelines`. Enabled by default. +> - Enforcement of scan execution policies on projects with an existing GitLab CI/CD configuration [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6880) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `scan_execution_policy_pipelines`. Feature flag `scan_execution_policy_pipelines` removed in GitLab 16.5. FLAG: On self-managed GitLab, this feature is enabled by default. To disable it, ask an @@ -28,6 +28,9 @@ implicitly so that the policies can be enforced. This ensures policies enabling secret detection, static analysis, or other scanners that do not require a build in the project, are still able to execute and be enforced. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Enforcing scan execution policies on projects with no GitLab CI/CD configuration](https://www.youtube.com/watch?v=sUfwQQ4-qHs). + In the event of a job name collision, GitLab appends a hyphen and a number to the job name. GitLab increments the number until the name no longer conflicts with existing job names. If you create a policy at the group level, it applies to every child project or subgroup. You cannot edit a @@ -96,9 +99,8 @@ the following sections and tables provide an alternative. ## `pipeline` rule type -> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Disabled by default. -> - Generally available in GitLab 16.2. Feature flag `security_policies_branch_type` removed. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. +> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Generally available in GitLab 16.2. Feature flag removed. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Generally available in GitLab 16.5. Feature flag removed. FLAG: On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. @@ -117,9 +119,11 @@ This rule enforces the defined actions whenever the pipeline runs for a selected ## `schedule` rule type -> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Disabled by default. -> - Generally available in GitLab 16.2. Feature flag `security_policies_branch_type` removed. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. +> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Generally available in GitLab 16.2. Feature flag removed. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Generally available in GitLab 16.5. Feature flag removed. + +WARNING: +In GitLab 16.1 and earlier, you should **not** use [direct transfer](../../../administration/settings/import_and_export_settings.md#enable-migration-of-groups-and-projects-by-direct-transfer) with scheduled scan execution policies. If using direct transfer, first upgrade to GitLab 16.2 and ensure security policy bots are enabled in the projects you are enforcing. FLAG: On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. @@ -141,7 +145,7 @@ This rule schedules a scan pipeline, enforcing the defined actions on the schedu Scheduled scan pipelines are triggered by a security policy bot user that is a guest member of the project with elevated permissions for users of type `security_policy_bot` so it may carry out this task. Security policy bot users are automatically created when the security policy project is linked, and removed when the security policy project is unlinked. -If the project does not have a security policy bot user, the scheduled scan pipeline will not be triggered. To recreate a security policy bot user unlink and link the security policy project again. +If the project does not have a security policy bot user, the bot will be automatically created, and the following scheduled scan pipeline will use it. GitLab supports the following types of CRON syntax for the `cadence` field: diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index a42b3f02c26..d892012c365 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -39,7 +39,7 @@ A project can have multiple pipeline types configured. A single commit can initi pipelines, each of which may contain a security scan. - In GitLab 16.3 and later, the results of all completed pipelines for the latest commit in -the MR's source and target branch are evaluated and used to enforce the scan result policy. +the merge request's source and target branch are evaluated and used to enforce the scan result policy. Parent-child pipelines and on-demand DAST pipelines are not considered. - In GitLab 16.2 and earlier, only the results of the latest completed pipeline were evaluated when enforcing scan result policies. @@ -101,14 +101,9 @@ On GitLab.com, this feature is not available. ## `scan_finding` rule type -> - The scan result policy field `vulnerability_attributes` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123052) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `enforce_vulnerability_attributes_rules`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/418784) in GitLab 16.3. +> - The scan result policy field `vulnerability_attributes` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123052) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `enforce_vulnerability_attributes_rules`. [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/418784) in GitLab 16.3. Feature flag `enforce_vulnerability_attributes_rules` removed in GitLab 16.5. > - The scan result policy field `vulnerability_age` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123956) in GitLab 16.2. -> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Enabled by default. - -FLAG: -On self-managed GitLab, by default the `vulnerability_attributes` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `enforce_vulnerability_attributes_rules`. -On GitLab.com, this feature is available. +> - The `branch_exceptions` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418741) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. Generally available in GitLab 16.5. Feature flag removed. FLAG: On self-managed GitLab, by default the `branch_exceptions` field is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `security_policies_branch_exceptions`. diff --git a/doc/user/application_security/sast/rules.md b/doc/user/application_security/sast/rules.md index 4e7a6387f9b..e4054764e1f 100644 --- a/doc/user/application_security/sast/rules.md +++ b/doc/user/application_security/sast/rules.md @@ -38,6 +38,18 @@ Analyzers and their rules are updated [at least monthly](../index.md#vulnerabili The GitLab ruleset for the Semgrep-based analyzer is managed in [the GitLab-managed open-source `sast-rules` project](https://gitlab.com/gitlab-org/security-products/sast-rules). When rules are updated, they're released as part of the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep)'s container image. +### Rule update policies + +Updates to SAST rules are not [breaking changes](../../../update/terminology.md#breaking-change). +This means that rules may be added, removed, or updated without prior notice. + +However, to make rule changes more convenient and understandable, GitLab: + +- Documents [rule changes](#important-rule-changes) that are planned or completed. +- [Automatically resolves](index.md#automatic-vulnerability-resolution) findings from rules after they are removed for Semgrep-based analyzers. +- Enables you to [change the status on vulnerabilities where activity = "no longer detected" in bulk](../vulnerability_report/index.md#change-status-of-vulnerabilities). +- Evaluates proposed rule changes for the impact they will have on existing vulnerability records. + ## Configure rules in your projects You should use the default SAST rules unless you have a specific reason to make a change. diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index a10c029bec6..18016f6f342 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -15,6 +15,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > `secret_detection_default_branch` and `secret_detection` were consolidated into one job, > `secret_detection`. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo of this Secret Detection doc, see [How to enable secret detection in GitLab Application Security Part 1/2](https://youtu.be/dbMxeO6nJCE?feature=shared) and [How to enable secret detection in GitLab Application Security Part 2/2](https://youtu.be/VL-_hdiTazo?feature=shared) +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an interactive reading and how-to demo playlist, see [Get Started With GitLab Application Security Playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KrUrjDoefSkgZLx5aJYFaF9) + People sometimes accidentally commit secrets like keys or API tokens to Git repositories. After a sensitive value is pushed to a remote repository, anyone with access to the repository can impersonate the authorized user of the secret for malicious purposes. Most organizations require exposed secrets to be revoked and replaced to address this risk. diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index 5c2dbd8d728..0f0a61a2b02 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -35,6 +35,11 @@ The different places in an application that are vulnerable to attack. Secure pro search the attack surface during scans. Each product defines the attack surface differently. For example, SAST uses files and line numbers, and DAST uses URLs. +## Component + +A software component that makes up a portion of a software project. Examples include libraries, drivers, data, and +[many more](https://cyclonedx.org/docs/1.5/json/#components_items_type). + ## Corpus The set of meaningful test cases that are generated while the fuzzer is running. Each meaningful @@ -105,6 +110,12 @@ under development and tracked in issue [267588](https://gitlab.com/gitlab-org/gi A legitimate finding that a particular customer doesn't care about. +## Known affected component + +A component that matches the requirements for a vulnerability to be exploitable. For example, +`packageA@1.0.3` matches the name, package type, and one of the affected versions or version +ranges of `FAKECVE-2023-0001`. + ## Location fingerprint A finding's location fingerprint is a text value that's unique for each location on the attack @@ -217,6 +228,14 @@ table.package-managers-and-types ul { A page that displays findings discovered in the associated CI pipeline. +## Possibly affected component + +A software component that is possibly affected by vulnerability. For example, when scanning a +project for known vulnerabilities, components are first evaluated to see if they match the name +and [package type](https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst). +During this stage, they're _possibly_ affected by the vulnerability, and are only [known to be affected](#known-affected-component) +after it's confirmed that they fall in the affected version range. + ## Post-filter Post-filters help reduce noise in the scanner results and automate manual tasks. You can specify criteria that updates diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index 31946d414a2..db6b4e0f4a8 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -34,7 +34,7 @@ Medium severity vulnerabilities usually arise from misconfiguration of systems o ## Low severity -Low severity vulnerabilities contain flaws that may not be directly exploitable but introduce unnecessary weakness to an application or system. These flaws are usually due to missing security controls, or unnecessary disclose information about the application environment. Examples of low severity vulnerabilities are missing cookie security directives, verbose error or exception messages. Typically these flaws are rated with CVSS 3.1 between 1.0-3.9. +Low severity vulnerabilities contain flaws that may not be directly exploitable but introduce unnecessary weakness to an application or system. These flaws are usually due to missing security controls, or unnecessary disclose information about the application environment. Examples of low severity vulnerabilities are missing cookie security directives, verbose error or exception messages. Typically these flaws are rated with CVSS 3.1 between 0.1-3.9. ## Info severity diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 3ec1151b1d6..24ed318e688 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -67,6 +67,7 @@ From the Vulnerability Report you can: - [Export details of vulnerabilities](#export-vulnerability-details). - [Sort vulnerabilities by date](#sort-vulnerabilities-by-date-detected). - [Manually add a vulnerability finding](#manually-add-a-vulnerability-finding). +- [Grouping vulnerability report](#group-vulnerabilities) ## Vulnerability Report filters @@ -256,6 +257,19 @@ To add a new vulnerability finding from your project level Vulnerability Report You are brought to the newly created vulnerability's detail page. Manually created records appear in the Group, Project, and Security Center Vulnerability Reports. To filter them, use the Generic Tool filter. +## Group vulnerabilities + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420055) in GitLab 16.4. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/422509) in GitLab 16.5. + +To group the Vulnerability Report: + +1. Below the **Vulnerability Report** filters, select the **Group By** dropdown list. +1. Select the attribute you want to group by: status or severity. + +To see what is included in a group, select a category to expand the report and see related +vulnerabilities. + ## Operational vulnerabilities > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6345) in GitLab 14.6. diff --git a/doc/user/award_emojis.md b/doc/user/award_emojis.md index 26cccd7584e..09f7b4c77fa 100644 --- a/doc/user/award_emojis.md +++ b/doc/user/award_emojis.md @@ -1,68 +1,11 @@ --- -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'emoji_reactions.md' +remove_date: '2023-12-20' --- -# Emoji reactions **(FREE ALL)** +This document was moved to [another location](emoji_reactions.md). -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emoji" to "emoji reactions" in GitLab 16.0. -> - Reacting with emoji on work items (such as tasks, objectives, and key results) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393599) in GitLab 16.0. -> - Reacting with emoji on design discussion comments [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29756) in GitLab 16.2. - -When you're collaborating online, you get fewer opportunities for high-fives -and thumbs-ups. React with emoji on: - -- [Issues](project/issues/index.md). -- [Tasks](tasks.md). -- [Merge requests](project/merge_requests/index.md), -[snippets](snippets.md). -- [Epics](../user/group/epics/index.md). -- [Objectives and key results](okrs.md). -- Anywhere else you can have a comment thread. - -![Emoji reactions](img/award_emoji_select_v14_6.png) - -Emoji reactions make it much easier to give and receive feedback without a long -comment thread. - -"Thumbs up" and "thumbs down" emoji are used to calculate an issue or merge request's position when -[sorting by popularity](project/issues/sorting_issue_lists.md#sorting-by-popularity). - -For information on the relevant API, see [Emoji reactions API](../api/award_emoji.md). - -## Emoji reactions for comments - -Emoji reactions can also be applied to individual comments when you want to -celebrate an accomplishment or agree with an opinion. - -To add an emoji reaction: - -1. In the upper-right corner of the comment, select the smile (**{slight-smile}**). -1. Select an emoji from the emoji picker. - -To remove an emoji reaction, select the emoji again. - -## Custom emoji - -> - [Introduced for GraphQL API](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6 [with a flag](../administration/feature_flags.md) named `custom_emoji`. Disabled by default. -> - Enabled on GitLab.com in GitLab 14.0. -> - UI to add emoji [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333095) in GitLab 16.2. - -Custom emoji show in the emoji picker everywhere you can react with emoji. -To add an emoji reaction to a comment or description: - -1. Select **Add reaction** (**{slight-smile}**). -1. Select the GitLab logo (**{tanuki}**) or scroll down to the **Custom** section. -1. Select an emoji from the emoji picker. - -![Custom emoji in emoji picker](img/custom_emoji_reactions_v16_2.png) - -To use them in a text box, type the filename between two colons. -For example, `:thank-you:`. - -You can upload custom emoji to a GitLab instance with the GraphQL API. -For more information, see [Use custom emoji with GraphQL](../api/graphql/custom_emoji.md). - -For a list of custom emoji available for GitLab.com, see -[the `custom_emoji` project](https://gitlab.com/custom_emoji/custom_emoji/-/tree/main/img). +<!-- This redirect file can be deleted after <2023-12-20>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 3c5c90abdae..f2f9aceda69 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -99,9 +99,8 @@ to a management project: | Staging | `staging` | | Production | `production` | -The following environments set in -[`.gitlab-ci.yml`](../../ci/yaml/index.md) deploy to the -Development, Staging, and Production cluster respectively. +The environments set in the +[`.gitlab-ci.yml`](../../ci/yaml/index.md) file deploy to the Development, Staging, and Production cluster. ```yaml stages: diff --git a/doc/user/compliance/compliance_center/index.md b/doc/user/compliance/compliance_center/index.md index 2510b5e73a7..0e205a29920 100644 --- a/doc/user/compliance/compliance_center/index.md +++ b/doc/user/compliance/compliance_center/index.md @@ -15,12 +15,12 @@ See report and manage standards adherence, violations, and compliance frameworks > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125875) GraphQL APIs in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `compliance_adherence_report`. Disabled by default. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125444) standards adherence dashboard in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `adherence_report_ui`. Disabled by default. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/414495) in GitLab 16.5. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire -instance, an administrator can [enable the feature flags](../../../administration/feature_flags.md) named -`compliance_adherence_report` and `adherence_report_ui`. On GitLab.com, this feature is not available. -This feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, an administrator can +[disable the feature flags](../../../administration/feature_flags.md) named `compliance_adherence_report` and `adherence_report_ui`. On GitLab.com, +this feature is available. Standards adherence dashboard lists the adherence status of projects complying to GitLab standard. @@ -167,6 +167,33 @@ separation of duties is: - [A merge request committer is **not** allowed to approve a merge request they have added commits to](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). - [The minimum number of approvals required to merge a merge request is **at least** two](../../project/merge_requests/approvals/rules.md). +### Export a report of merge request compliance violations on projects in a group + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356791) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `compliance_violation_csv_export`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/424447) in GitLab 16.5. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named +`compliance_violation_csv_export`. On GitLab.com, this feature is available. + +Export a report of merge request compliance violations on merge requests belonging to projects in a group. Reports: + +- Do not use filters on the violations report. +- Are truncated at 15 MB so the email attachment is not too large. + +Prerequisites: + +- You must be an administrator or have the Owner role for the group. + +To export a report of merge request compliance violations for projects in a group: + +1. On the left sidebar, select **Search or go to** and find your group. +1. On the left sidebar, select **Secure > Compliance center**. +1. On the page, select the **Violations** tab. +1. On the Violations tab, select the **Export full report as CSV** action in the top right corner + +A report is compiled and delivered to your email inbox as an attachment. + ### Chain of Custody report > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213364) in GitLab 13.3. @@ -234,11 +261,12 @@ Depending on your version of GitLab, the Chain of Custody report is either sent Alternatively, use a direct link: `https://gitlab.com/groups/<group-name>/-/security/merge_commit_reports.csv?commit_sha={optional_commit_sha}`, passing in an optional value to the `commit_sha` query parameter. -## Compliance frameworks report +## Compliance projects report -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387910) in GitLab 15.10. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387910) in GitLab 15.10. +> - [Renamed from **compliance frameworks report**](https://gitlab.com/gitlab-org/gitlab/-/issues/422963) in GitLab 16.5. -With compliance frameworks report, you can see the compliance frameworks that are applied to projects in a group. Each row of the report shows: +With compliance projects report, you can see the compliance frameworks that are applied to projects in a group. Each row of the report shows: - Project name. - Project path. @@ -246,17 +274,17 @@ With compliance frameworks report, you can see the compliance frameworks that ar The default framework for the group has a **default** badge. -### View the compliance frameworks report for a group +### View the compliance projects report for a group Prerequisites: - You must be an administrator or have the Owner role for the group. -To view the compliance frameworks report: +To view the compliance projects report: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. ### Apply a compliance framework to projects in a group @@ -273,7 +301,7 @@ To apply a compliance framework to one project in a group: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. Next to the project you want to add the compliance framework to, select **{plus}** **Add framework**. 1. Select an existing compliance framework or create a new one. @@ -281,7 +309,7 @@ To apply a compliance framework to multiple projects in a group: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. Select multiple projects. 1. From the **Choose one bulk action** dropdown list, select **Apply framework to selected projects**. 1. Select framework to apply. @@ -302,44 +330,18 @@ To remove a compliance framework from one project in a group: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. Next to the compliance framework to remove from the project, select **{close}** on the framework label. To remove a compliance framework from multiple projects in a group: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. Select multiple projects. 1. From the **Choose one bulk action** dropdown list, select **Remove framework from selected projects**. 1. Select **Remove**. -### Export a report of merge request compliance violations on projects in a group - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356791) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `compliance_violation_csv_export`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`compliance_violation_csv_export`. On GitLab.com, this feature is not available. The feature is not ready for production use. - -Export a report of merge request compliance violations on merge requests belonging to projects in a group. Reports: - -- Do not use filters on the violations report. -- Are truncated at 15 MB so the email attachment is not too large. - -Prerequisites: - -- You must be an administrator or have the Owner role for the group. - -To export a report of merge request compliance violations for projects in a group: - -1. On the left sidebar, select **Search or go to** and find your group. -1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Violations** tab. -1. On the Violations tab, select the **Export full report as CSV** action in the top right corner - -A report is compiled and delivered to your email inbox as an attachment. - ### Export a report of compliance frameworks on projects in a group > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387912) in GitLab 16.0. @@ -357,12 +359,12 @@ To export a report of compliance frameworks on projects in a group: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. On the Frameworks tab, select the **Export as CSV** action in the top right corner A report is compiled and delivered to your email inbox as an attachment. -#### Filter the compliance frameworks report +#### Filter the compliance projects report > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387911) in GitLab 15.11. @@ -370,7 +372,7 @@ To filter the list of compliance frameworks: 1. On the left sidebar, select **Search or go to** and find your group. 1. On the left sidebar, select **Secure > Compliance center**. -1. On the page, select the **Frameworks** tab. +1. On the page, select the **Projects** tab. 1. In the search field: 1. Select the attribute you want to filter by. 1. Select an operator. @@ -378,3 +380,30 @@ To filter the list of compliance frameworks: 1. Select **Search** (**{search}**). Repeat this process to filter by multiple attributes. + +## Compliance frameworks report + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422973) in GitLab 16.5 [with a flag](../../../administration/feature_flags.md) named `compliance_framework_report_ui`. Disabled by default. +> - In GitLab 16.4 and earlier, **Compliance frameworks report** referred to what is now called **Compliance projects report**. The formally-named **Compliance frameworks report** was [renamed to **Compliance projects report**](https://gitlab.com/gitlab-org/gitlab/-/issues/422963) in GitLab 16.5. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named +`compliance_framework_report_ui`. On GitLab.com, this feature is not available. The feature is not ready for production use. + +With compliance frameworks report, you can see all the compliance frameworks in a group. Each row of the report shows: + +- Framework name. + +The default framework for the group has a **default** badge. + +### View the compliance frameworks report for a group + +Prerequisites: + +- You must be an administrator or have the Owner role for the group. + +To view the compliance projects report: + +1. On the left sidebar, select **Search or go to** and find your group. +1. On the left sidebar, select **Secure > Compliance center**. +1. On the page, select the **Frameworks** tab. diff --git a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md index 98404ecd2ed..81f7cc61782 100644 --- a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md +++ b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md @@ -22,8 +22,11 @@ Licenses not in the SPDX list are reported as "Unknown". License information can ## Configuration -Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration) -and ensure that its prerequisites are met. +Prerequisites: + +- On GitLab self-managed only, enable [Synchronization with the GitLab License Database](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in the Admin Area for the GitLab instance. On GitLab SaaS this step has already been completed. +- Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#enabling-the-analyzer) + and ensure that its prerequisites are met. From the `.gitlab-ci.yml` file, remove the deprecated line `Jobs/License-Scanning.gitlab-ci.yml`, if it's present. @@ -184,6 +187,7 @@ To remove the unneeded data: 1. If there is deprecated data in the database, remove it by running the following commands in order: ```ruby + ActiveRecord::Base.connection.execute('SET statement_timeout TO 0') PackageMetadata::PackageVersionLicense.delete_all PackageMetadata::PackageVersion.delete_all ``` diff --git a/doc/user/custom_roles.md b/doc/user/custom_roles.md new file mode 100644 index 00000000000..a13c45306ad --- /dev/null +++ b/doc/user/custom_roles.md @@ -0,0 +1,189 @@ +--- +stage: Govern +group: Authentication +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Custom roles **(ULTIMATE ALL)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106256) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `customizable_roles`. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110810) in GitLab 15.9. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114524) in GitLab 15.10. +> - The ability for a custom role to view a vulnerability report [introduced](https://gitlab.com/groups/gitlab-org/-/epics/10160) in GitLab 16.1 [with a flag](../administration/feature_flags.md) named `custom_roles_vulnerability`. +> - Ability to view a vulnerability report [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123835) in GitLab 16.1. +> - [Feature flag `custom_roles_vulnerability` removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124049) in GitLab 16.2. +> - Ability to create and remove a custom role with the UI [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393235) in GitLab 16.4. +> - Ability to manage group members [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17364) in GitLab 16.5 under `admin_group_member` Feature flag. +> - Ability to manage project access tokens [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/421778) in GitLab 16.5 under `manage_project_access_tokens` Feature flag. + +Custom roles allow group members who are assigned the Owner role to create roles +specific to the needs of their organization. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo of the custom roles feature, see [[Demo] Ultimate Guest can view code on private repositories via custom role](https://www.youtube.com/watch?v=46cp_-Rtxps). + +The following granular permissions are available. You can add these permissions to any base role, and add them in combination with each other to create a customized role: + +- The Guest+1 role, which allows users with the Guest role to view code. +- In GitLab 16.1 and later, you can create a custom role that can view vulnerability reports and change the status of the vulnerabilities. +- In GitLab 16.3 and later, you can create a custom role that can view the dependency list. +- In GitLab 16.4 and later, you can create a custom role that can approve merge requests. +- In GitLab 16.5 and later, you can create a custom role that can manage group members. + +You can discuss individual custom role and permission requests in [issue 391760](https://gitlab.com/gitlab-org/gitlab/-/issues/391760). + +When you enable a custom role for a user with the Guest role, that user has +access to elevated permissions, and therefore: + +- Is considered a [billable user](../subscriptions/self_managed/index.md#billable-users) on self-managed GitLab. +- [Uses a seat](../subscriptions/gitlab_com/index.md#how-seat-usage-is-determined) on GitLab.com. + +This does not apply to Guest+1, a Guest custom role that only enables the `read_code` +permission. Users with that specific custom role are not considered billable users +and do not use a seat. + +## Create a custom role + +Prerequisites: + +- You must be an administrator for the self-managed instance, or have the Owner + role in the group you are creating the custom role in. +- The group must be in the Ultimate tier. +- You must have: + - At least one private project so that you can see the effect of giving a + user with the Guest role a custom role. The project can be in the group itself + or one of that group's subgroups. + - A [personal access token with the API scope](profile/personal_access_tokens.md#create-a-personal-access-token). + +### GitLab SaaS + +Prerequisite: + +- You must have the Owner role in the group you are creating the custom role in. + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Roles and Permissions**. +1. Select **Add new role**. +1. In **Base role to use as template**, select **Guest**. +1. In **Role name**, enter the custom role's title. +1. Select the **Permissions** for the new custom role. +1. Select **Create new role**. + +### Self Managed GitLab Instances + +Prerequisite: + +- You must be an administrator for the self-managed instance you are creating the custom role in. + +1. On the left sidebar, select **Search or go to**. +1. Select **Admin Area**. +1. Select **Settings > Roles and Permissions**. +1. From the top dropdown list, select the group you want to create a custom role in. +1. Select **Add new role**. +1. In **Base role to use as template**, select **Guest**. +1. In **Role name**, enter the custom role's title. +1. Select the **Permissions** for the new custom role. +1. Select **Create new role**. + +To create a custom role, you can also [use the API](../api/member_roles.md#add-a-member-role-to-a-group). + +### Custom role requirements + +For every ability, a minimal access level is defined. To be able to create a custom role which enables a certain ability, the `member_roles` table record has to have the associated minimal access level. For all abilities, the minimal access level is Guest. Only users who have at least the Guest role can be assigned to a custom role. + +Some roles and abilities require having other abilities enabled. For example, a custom role can only have administration of vulnerabilities (`admin_vulnerability`) enabled if reading vulnerabilities (`read_vulnerability`) is also enabled. + +You can see the abilities requirements in the following table. + +| Ability | Required ability | +| -- | -- | +| `read_code` | - | +| `read_dependency` | - | +| `read_vulnerability` | - | +| `admin_merge_request` | - | +| `admin_vulnerability` | `read_vulnerability` | +| `admin_group_member` | - | +| `manage_project_access_tokens` | - | + +## Associate a custom role with an existing group member + +To associate a custom role with an existing group member, a group member with +the Owner role: + +1. Invites a user as a direct member to the root group or any subgroup or project in the root + group's hierarchy as a Guest. At this point, this Guest user cannot see any + code on the projects in the group or subgroup. +1. Optional. If the Owner does not know the `id` of the Guest user receiving a custom + role, finds that `id` by making an [API request](../api/member_roles.md#list-all-member-roles-of-a-group). + +1. Associates the member with the Guest+1 role using the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) + + ```shell + # to update a project membership + curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": '<member_role_id>', "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" + + # to update a group membership + curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": '<member_role_id>', "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" + ``` + + Where: + + - `<project_id` and `<group_id>`: The `id` or [URL-encoded path of the project or group](../api/rest/index.md#namespaced-path-encoding) associated with the membership receiving the custom role. + - `<member_role_id>`: The `id` of the member role created in the previous section. + - `<user_id>`: The `id` of the user receiving a custom role. + + Now the Guest+1 user can view code on all projects associated with this membership. + +## Remove a custom role + +Prerequisite: + +- You must be an administrator or have the Owner role in the group you are removing the custom role from. + +You can remove a custom role from a group only if no group members have that role. + +To do this, you can either remove the custom role from all group members with that custom role, or remove those members from the group. + +### Remove a custom role from a group member + +To remove a custom role from a group member, use the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) +and pass an empty `member_role_id` value. + +```shell +# to update a project membership +curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" + +# to update a group membership +curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" +``` + +### Remove a group member with a custom role from the group + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Manage > Members**. +1. On the member row you want to remove, select the vertical ellipsis + (**{ellipsis_v}**) and select **Remove member**. +1. In the **Remove member** confirmation dialog, do not select any checkboxes. +1. Select **Remove member**. + +### Delete the custom role + +After you have made sure no group members have that custom role, delete the +custom role. + +1. On the left sidebar, select **Search or go to**. +1. GitLab.com only. Select **Admin Area**. +1. Select **Settings > Roles and Permissions**. +1. Select **Custom Roles**. +1. In the **Actions** column, select **Delete role** (**{remove}**) and confirm. + +To delete a custom role, you can also [use the API](../api/member_roles.md#remove-member-role-of-a-group). +To use the API, you must know the `id` of the custom role. If you do not know this +`id`, find it by making an [API request](../api/member_roles.md#list-all-member-roles-of-a-group). + +## Known issues + +- If a user with a custom role is shared with a group or project, their custom + role is not transferred over with them. The user has the regular Guest role in + the new group or project. +- You cannot use an [Auditor user](../administration/auditor_users.md) as a template for a custom role. diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 9d4a151cc53..ae74b534e02 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -199,7 +199,7 @@ You can also mark an [issue as confidential](../project/issues/confidential_issu ## Show only comments In discussions with many comments, filter the discussion to show only comments or history of -changes (system notes). System notes include changes to the description, mentions in other GitLab +changes ([system notes](../project/system_notes.md)). System notes include changes to the description, mentions in other GitLab objects, or changes to labels, assignees, and the milestone. GitLab saves your preference, and applies it to every issue, merge request, or epic you view. diff --git a/doc/user/emoji_reactions.md b/doc/user/emoji_reactions.md new file mode 100644 index 00000000000..1b2c1bcd2e5 --- /dev/null +++ b/doc/user/emoji_reactions.md @@ -0,0 +1,68 @@ +--- +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Emoji reactions **(FREE ALL)** + +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emoji" to "emoji reactions" in GitLab 16.0. +> - Reacting with emoji on work items (such as tasks, objectives, and key results) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393599) in GitLab 16.0. +> - Reacting with emoji on design discussion comments [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29756) in GitLab 16.2. + +When you're collaborating online, you get fewer opportunities for high-fives +and thumbs-ups. React with emoji on: + +- [Issues](project/issues/index.md). +- [Tasks](tasks.md). +- [Merge requests](project/merge_requests/index.md), +[snippets](snippets.md). +- [Epics](../user/group/epics/index.md). +- [Objectives and key results](okrs.md). +- Anywhere else you can have a comment thread. + +![Emoji reactions](img/award_emoji_select_v14_6.png) + +Emoji reactions make it much easier to give and receive feedback without a long +comment thread. + +"Thumbs up" and "thumbs down" emoji are used to calculate an issue or merge request's position when +[sorting by popularity](project/issues/sorting_issue_lists.md#sorting-by-popularity). + +For information on the relevant API, see [Emoji reactions API](../api/emoji_reactions.md). + +## Emoji reactions for comments + +Emoji reactions can also be applied to individual comments when you want to +celebrate an accomplishment or agree with an opinion. + +To add an emoji reaction: + +1. In the upper-right corner of the comment, select the smile (**{slight-smile}**). +1. Select an emoji from the emoji picker. + +To remove an emoji reaction, select the emoji again. + +## Custom emoji + +> - [Introduced for GraphQL API](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6 [with a flag](../administration/feature_flags.md) named `custom_emoji`. Disabled by default. +> - Enabled on GitLab.com in GitLab 14.0. +> - UI to add emoji [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333095) in GitLab 16.2. + +Custom emoji show in the emoji picker everywhere you can react with emoji. +To add an emoji reaction to a comment or description: + +1. Select **Add reaction** (**{slight-smile}**). +1. Select the GitLab logo (**{tanuki}**) or scroll down to the **Custom** section. +1. Select an emoji from the emoji picker. + +![Custom emoji in emoji picker](img/custom_emoji_reactions_v16_2.png) + +To use them in a text box, type the filename between two colons. +For example, `:thank-you:`. + +You can upload custom emoji to a GitLab instance with the GraphQL API. +For more information, see [Use custom emoji with GraphQL](../api/graphql/custom_emoji.md). + +For a list of custom emoji available for GitLab.com, see +[the `custom_emoji` project](https://gitlab.com/custom_emoji/custom_emoji/-/tree/main/img). diff --git a/doc/user/enterprise_user/index.md b/doc/user/enterprise_user/index.md index 04683620ba9..2909c06046e 100644 --- a/doc/user/enterprise_user/index.md +++ b/doc/user/enterprise_user/index.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -13,29 +13,58 @@ has purchased a [GitLab subscription](../../subscriptions/index.md). Enterprise users are identified by the **Enterprise** badge next to their names on the [Members list](../group/index.md#filter-and-sort-members-in-a-group). -## Provision an enterprise user +## Automatic claims of enterprise users -A user account is considered an enterprise account when: +A user is automatically claimed as an enterprise user of a group when **all** of the following conditions are met: -- A user without an existing GitLab user account uses the group's - [SAML SSO](../group/saml_sso/index.md) to sign in for the first time. -- [SCIM](../group/saml_sso/scim_setup.md) creates the user account on behalf of - the group. +1. The user's primary email has a domain that has been [verified](#verified-domains-for-groups) by the paid group. +1. The user account meets at least **one** of the following conditions: + - It was created February 1, 2021 or later. + - It has a SAML or SCIM identity tied to the organization's group. + - It has a `provisioned_by_group_id` value that is the same as the organization's group's ID. + - It is a member of the organization's group, where the subscription was purchased or renewed February 1, 2021 or later. -A user can also [manually connect an identity provider (IdP) to a GitLab account whose email address matches the subscribing organization's domain](../group/saml_sso/index.md#link-saml-to-your-existing-gitlabcom-account). -By selecting **Authorize** when connecting these two accounts, the user account -with the matching email address is classified as an enterprise user. However, this -user account does not have an **Enterprise** badge in GitLab, and a group Owner cannot -disable the user's two-factor authentication. +After the user is claimed as an enterprise user: -Although a user can be a member of more than one group, each user account can be -provisioned by only one group. As a result, a user is considered an enterprise -user under one top-level group only. +- Their `enterprise_group_id` attribute is set to the organization's group's ID. +- The user receives a [welcome email](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/views/notify/user_associated_with_enterprise_group_email.html.haml). + +If a group's purchased subscription expires or is canceled: + +- Users claimed as enterprise users remain enterprise users of that group. +- The group is not able to [manage their enterprise users](#manage-enterprise-users-in-a-namespace). +- [Enterprise user restrictions](#enterprise-user-restrictions) apply to those user accounts. +- No new users can be [automatically associated with the group](#automatic-claims-of-enterprise-users) until the paid subscription is renewed. + +If a group's verified domains are removed: + +- Users claimed as enterprise users remain enterprise users of that group. +- [Enterprise user restrictions](#enterprise-user-restrictions) apply to those user accounts. +- No new users can be [automatically associated with the group](#automatic-claims-of-enterprise-users) until domains are verified. + +If the organization moves its verified domains to another paid group, its enterprise users are [automatically claimed](#automatic-claims-of-enterprise-users) as enterprise users of that group. + +## Enterprise user restrictions + +### Primary email change + +An enterprise user can only change their primary email to an email their organization owns as per its verified domains. +If an organization removes all its verified domains, its enterprise users are not able to change their primary email address. + +Only GitLab administrators can change enterprise users' primary email address to an email with a non-verified domain. + +Providing the ability to group Owners to change their enterprise users' primary email to an email with a non-verified domain is proposed in [issue 412966](https://gitlab.com/gitlab-org/gitlab/-/issues/412966). + +## Dissociation of the user from their enterprise group + +Changing an enterprise user's primary email to an email with a non-verified domain automatically disassociates them from their enterprise group. +However, there are [primary email change restrictions](#primary-email-change). ## Verified domains for groups The following automated processes use [verified domains](../project/pages/custom_domains_ssl_tls_certification/index.md) to run: +- [Automatic claims of enterprise users](#automatic-claims-of-enterprise-users). - [Bypass email confirmation for provisioned users](#bypass-email-confirmation-for-provisioned-users). ### Set up a verified domain diff --git a/doc/user/free_push_limit.md b/doc/user/free_push_limit.md new file mode 100644 index 00000000000..c0b23720ab1 --- /dev/null +++ b/doc/user/free_push_limit.md @@ -0,0 +1,47 @@ +--- +stage: Growth +group: Acquisition +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Free push limit **(FREE SAAS)** + +A 100 MB per-file limit applies when pushing new files to any project in the Free tier. + +If a new file that is 100 MB or large is pushed to a project in the Free tier, an error is displayed. For example: + +```shell +Enumerating objects: 3, done. +Counting objects: 100% (3/3), done. +Delta compression using up to 10 threads +Compressing objects: 100% (2/2), done. +Writing objects: 100% (3/3), 100.03 MiB | 1.08 MiB/s, done. +Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 +remote: GitLab: You are attempting to check in one or more files which exceed the 100MiB limit: + +- 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 (123 MiB) +- 5716ca5987cbf97d6bb54920bea6adde242d87e6 (396 MiB) + +Please refer to https://docs.gitlab.com/ee/user/free_user_limit.html for further information. +To https://gitlab.com/group/my-project.git + ! [remote rejected] main -> main (pre-receive hook declined) +error: failed to push some refs to 'https://gitlab.com/group/my-project.git' +``` + +The error lists the unique IDs for files rather than their filename. To look up the file name from the unique identify, run the following command: + +```shell +tree -r | grep <id> +``` + +Because Git is not designed to handle large non-text-based data well, you should use [Git LFS](../topics/git/lfs/index.md) for these files. +Git LFS is designed to work with Git to track large files. + +## Feedback + +If you have any feedback to share about this limit, please do so in +[issue 428188](https://gitlab.com/gitlab-org/gitlab/-/issues/428188). + +## Related topics + +- [GitLab SaaS Free tier frequently asked questions](https://about.gitlab.com/pricing/faq-efficient-free-tier/). diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 09e05538fd7..6f809ae867a 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -210,21 +210,22 @@ varies by format: GitLab.com has the following account limits enabled. If a setting is not listed, the default value [is the same as for self-managed instances](../../administration/settings/account_and_limit_settings.md): -| Setting | GitLab.com default | -|--------------------------------------------------------------------------------------------------------------------|--------------------| -| [Repository size including LFS](../../administration/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | -| [Maximum import size](../project/settings/import_export.md#import-a-project-and-its-data) | 5 GB | -| Maximum remote file size for imports from external object storages | 10 GB | -| Maximum download file size when importing from source GitLab instances by direct transfer | 5 GB | -| Maximum attachment size | 100 MB | -| [Maximum decompressed file size for imported archives](../../administration/settings/import_and_export_settings.md#maximum-decompressed-file-size-for-imported-archives) | 25 GB | - -If you are near or over the repository size limit, you can either -[reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md) -or [purchase additional storage](https://about.gitlab.com/pricing/licensing-faq/#can-i-buy-more-storage). +| Setting | GitLab.com default | +|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------| +| [Repository size including LFS](../../administration/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | +| [Maximum import size](../project/settings/import_export.md#import-a-project-and-its-data) | 5 GiB | +| [Maximum remote file size for imports from external object storages](../../administration/settings/import_and_export_settings.md#maximum-remote-file-size-for-imports) | 10 GiB | +| [Maximum download file size when importing from source GitLab instances by direct transfer](../../administration/settings/import_and_export_settings.md#maximum-download-file-size-for-imports-by-direct-transfer) | 5 GiB | +| Maximum attachment size | 100 MiB | +| [Maximum decompressed file size for imported archives](../../administration/settings/import_and_export_settings.md#maximum-decompressed-file-size-for-imported-archives) | 25 GiB | + +If you are near or over the repository size limit, you can either: + +- [Reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). +- [Purchase additional storage](https://about.gitlab.com/pricing/licensing-faq/#can-i-buy-more-storage). NOTE: -`git push` and GitLab project imports are limited to 5 GB per request through +`git push` and GitLab project imports are limited to 5 GiB per request through Cloudflare. Imports other than a file upload are not affected by this limit. Repository limits apply to both public and private projects. diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index 428c87143f6..966945b6b12 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -134,9 +134,11 @@ can be added to the group. The most popular public email domains cannot be restricted, such as: -- `gmail.com`, `yahoo.com`, `aol.com`, `icloud.com` -- `hotmail.com`, `hotmail.co.uk`, `hotmail.fr` -- `msn.com`, `live.com`, `outlook.com` +- `aol.com`, `gmail.com`, `hotmail.co.uk`, `hotmail.com`, +- `hotmail.fr`, `icloud.com`, `live.com`, `mail.com`, +- `me.com`, `msn.com`, `outlook.com`, +- `proton.me`, `protonmail.com`, `tutanota.com`, +- `yahoo.com`, `yandex.com`, `zohomail.com` When you share a group, both the source and target namespaces must allow the domains of the members' email addresses. @@ -188,7 +190,7 @@ prevent a project from being shared with other groups: 1. Select **Projects in `<group_name>` cannot be shared with other groups**. 1. Select **Save changes**. -This setting applies to all subgroups unless overridden by a group Owner. Groups already +This setting, when enabled, applies to all subgroups unless overridden by a group Owner. Groups already added to a project lose access when the setting is enabled. ## Prevent users from requesting access to a group diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md index 87c1c548abd..967ba4e05d0 100644 --- a/doc/user/group/custom_project_templates.md +++ b/doc/user/group/custom_project_templates.md @@ -94,7 +94,7 @@ to a user in the template are reassigned to you. It's important to understand this reassignment when you configure security features like protected branches and tags. For example, if the template contains a protected branch: -- In the template, the branch allows the _template owner_ to merge into the default branch. +- In the template, the branch allows the _template owner_ to merge into the default branch. - In the project created from the template, the branch allows _you_ to merge into the default branch. diff --git a/doc/user/group/epics/img/button_reopen_epic.png b/doc/user/group/epics/img/button_reopen_epic.png Binary files differdeleted file mode 100644 index 9d1be88549d..00000000000 --- a/doc/user/group/epics/img/button_reopen_epic.png +++ /dev/null diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 7b977dc2026..83d38dbc70b 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -66,7 +66,7 @@ have a start or due date, a visual - Link [related epics](linked_epics.md) based on a type of relationship. - Create workflows with [epic boards](epic_boards.md). - [Turn on notifications](../../profile/notifications.md) for about epic events. -- [Add an emoji reaction](../../award_emojis.md) to an epic or its comments. +- [Add an emoji reaction](../../emoji_reactions.md) to an epic or its comments. - Collaborate on an epic by posting comments in a [thread](../../discussions/index.md). - Use [health status](../../project/issues/managing_issues.md#health-status) to track your progress. diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index b79177e1571..5675393441e 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -159,11 +159,7 @@ Prerequisites: - You must have at least the Reporter role for the epic's group. -To close an epic, at the top of an epic, select **Close epic**. - -<!-- Delete when the `move_close_into_dropdown` feature flag is removed --> -If you don't see this action at the top of an epic, your project or instance might have -enabled a feature flag to [moved it in the actions menu](../../project/issues/managing_issues.md#move-the-close-button-into-the-actions-menu). +To close an epic, in the upper-right corner, select **epic actions** (**{ellipsis_v}**) and then **Close epic**. You can also use the `/close` [quick action](../../project/quick_actions.md). @@ -177,16 +173,9 @@ Prerequisites: To do so, either: -- Select **Reopen epic**. - - ![reopen epic - button](img/button_reopen_epic.png) - +- In the upper-right corner, select **epic actions** (**{ellipsis_v}**) and then **Reopen epic** - Use the `/reopen` [quick action](../../project/quick_actions.md). -<!-- Delete when the `move_close_into_dropdown` feature flag is removed --> -If you don't see this action at the top of an epic, your project or instance might have -enabled a feature flag to [moved it in the actions menu](../../project/issues/managing_issues.md#move-the-close-button-into-the-actions-menu). - ## Go to an epic from an issue If an issue belongs to an epic, you can go to the parent epic with the diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index a049b4afcc1..e1d5c8e5f0a 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -118,13 +118,18 @@ Hardcoded limits apply on migration by direct transfer. | Limit | Description | |:------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 6 | Maximum number of migrations permitted by a destination GitLab instance per minute per user. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386452) in GitLab 15.9. | -| 5 GB | Maximum relation size that can be downloaded from the source instance. | -| 10 GB | Maximum size of a decompressed archive. | | 210 seconds | Maximum number of seconds to wait for decompressing an archive file. | | 50 MB | Maximum length an NDJSON row can have. | | 5 minutes | Maximum number of seconds until an empty export status on source instance is raised. | | 8 hours | Time until migration times out. | +[Configurable limits](../../../administration/settings/account_and_limit_settings.md) are also available. + +In GitLab 16.3 and later, the following previously hard-coded settings are [configurable](https://gitlab.com/gitlab-org/gitlab/-/issues/384976): + +- Maximum relation size that can be downloaded from the source instance (set to 5 GiB). +- Maximum size of a decompressed archive (set to 10 GiB). + You can test the maximum relation size limit using these APIs: - [Group relations export API](../../../api/group_relations_export.md). diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index 65190847b05..d671b0434b6 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -130,7 +130,9 @@ After sharing the `Frontend` group with the `Engineering` group: - The **Groups** tab lists the `Engineering` group. - The **Groups** tab lists a group regardless of whether it is a public or private group. -- All direct members of the `Engineering` group have access to the `Frontend` group. Direct members of `Engineering` that gain access to the `Frontend` group keep their same access level as in `Engineering`, but up to the maximum access level selected when sharing the group. +- All direct members of the `Engineering` group have access to the `Frontend` group. The least access is granted between the access in the `Engineering` group and the access in the `Frontend` group. + - If `Member1` has the Maintainer role in `Engineering` and `Engineering` is added to `Frontend` with the Developer role, `Member1` has the Developer role in `Frontend`. + - If `Member2` has the Guest role in `Engineering` and `Engineering` is added to `Frontend` with the Developer role, `Member2` has the Guest role in `Frontend`. - Inherited members of the `Engineering` group do not gain access to the `Frontend` group. - Direct members of the `Engineering` group who have the **Group Invite** badge next to their profile on the group's usage quota page count towards the billable members of the `Frontend` group. @@ -271,7 +273,7 @@ You can remove the user cap, so there is no limit on the number of members you c Prerequisite: -- You must be assigned the Owner role) for the group. +- You must be assigned the Owner role for the group. To remove the user cap: @@ -452,9 +454,11 @@ You can give all users in a group and its subgroups access to [Code Suggestions] - This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) in the group. - Each user can - [Enable Code Suggestions](../../user/profile/preferences.md#enable-code-suggestions). + [enable Code Suggestions](../../user/profile/preferences.md#enable-code-suggestions). + +Code Suggestions are enabled by default at the group level. -To enable Code Suggestions for a group: +To update this setting: 1. On the left sidebar, select **Search or go to** and find your group. 1. Select **Settings > General**. @@ -462,16 +466,16 @@ To enable Code Suggestions for a group: 1. Under **Code Suggestions**, select the **Projects in this group can use Code Suggestions** checkbox. 1. Select **Save changes**. -## Enable Experiment features **(ULTIMATE SAAS)** +## Enable Experiment and Beta features **(ULTIMATE SAAS)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404856) in GitLab 16.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118222) in GitLab 16.0. WARNING: -[Experiment features](../../policy/experiment-beta-support.md#experiment) may produce unexpected results +[Experiment and Beta features](../../policy/experiment-beta-support.md) may produce unexpected results (for example, the results might be low-quality, incomplete, incoherent, offensive, or insensitive, and might include insecure code or failed pipelines). -You can give all users in a top-level group access to Experiment features. +You can give all users in a top-level group access to Experiment and Beta features. This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) that belong to the group. @@ -480,12 +484,12 @@ To enable Experiment features for a top-level group: 1. On the left sidebar, select **Search or go to** and find your group. 1. Select **Settings > General**. 1. Expand **Permissions and group features**. -1. Under **Experiment features**, select the **Use Experiment features** checkbox. +1. Under **Experiment and Beta features**, select the **Use Experiment and Beta features** checkbox. 1. Select **Save changes**. ## Enable third-party AI features **(ULTIMATE SAAS)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404856) in GitLab 16.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118222) in GitLab 16.0. WARNING: These AI features use [third-party services](../ai_features.md#data-usage) diff --git a/doc/user/group/saml_sso/example_saml_config.md b/doc/user/group/saml_sso/example_saml_config.md index 70e01e1d9a9..86ad2ba32d1 100644 --- a/doc/user/group/saml_sso/example_saml_config.md +++ b/doc/user/group/saml_sso/example_saml_config.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -50,7 +50,7 @@ Provisioning: ![Azure AD SCIM Provisioning](img/AzureAD-scim_provisioning.png) -Attribute mapping: +### Attribute mapping ![Azure AD SCIM Attribute Mapping](img/AzureAD-scim_attribute_mapping.png) @@ -70,15 +70,15 @@ If available, you can add user-friendly group names instead. When setting up Azu ## Google Workspace -Basic SAML app configuration: +### Basic SAML app configuration ![Google Workspace basic SAML](img/GoogleWorkspace-basic-SAML_v14_10.png) -User claims and attributes: +### User claims and attributes ![Google Workspace user claims](img/GoogleWorkspace-claims_v14_10.png) -IdP links and certificate: +### IdP links and certificate NOTE: Google Workspace displays a SHA256 fingerprint. To retrieve the SHA1 fingerprint required by GitLab for configuring SAML, download the certificate and calculate the SHA1 certificate @@ -88,53 +88,55 @@ fingerprint. ## Okta -Basic SAML app configuration for GitLab.com groups: +### Basic SAML app configuration for GitLab.com groups ![Okta basic SAML](img/Okta-GroupSAML.png) -Basic SAML app configuration for GitLab self-managed: +### Basic SAML app configuration for GitLab self-managed ![Okta admin panel view](img/Okta-SM.png) -User claims and attributes: +### User claims and attributes ![Okta Attributes](img/Okta-attributes.png) -Groups attribute: +### Group Sync ![Okta Group attribute](img/Okta-GroupAttribute.png) -Advanced SAML app settings (defaults): +### Advanced SAML app settings (defaults) ![Okta Advanced Settings](img/Okta-advancedsettings.png) -IdP Links and Certificate: +### IdP links and certificate ![Okta Links and Certificate](img/Okta-linkscert.png) -Sign on settings: +### SAML sign on settings ![Okta SAML settings](img/okta_saml_settings.png) +### SCIM settings + Setting the username for the newly provisioned users when assigning them the SCIM app: ![Assigning SCIM app to users on Okta](img/okta_setting_username.png) ## OneLogin -Application details: +### Basic SAML app configuration ![OneLogin application details](img/OneLogin-app_details.png) -Parameters: +### Parameters ![OneLogin application details](img/OneLogin-parameters.png) -Adding a user: +### Adding a user ![OneLogin user add](img/OneLogin-userAdd.png) -SSO settings: +### SSO settings ![OneLogin SSO settings](img/OneLogin-SSOsettings.png) diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md index 335c989c85f..c18ccaf9c20 100644 --- a/doc/user/group/saml_sso/group_sync.md +++ b/doc/user/group/saml_sso/group_sync.md @@ -1,7 +1,7 @@ --- type: reference, howto stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -69,9 +69,11 @@ For example, Azure AD sends the Azure Group Object ID instead of the name. Use t ``` Other attribute names such as `http://schemas.microsoft.com/ws/2008/06/identity/claims/groups` -are not accepted as a source of groups. For more information on configuring the -required attribute name in the SAML identity provider's settings, see -[example group SAML and SCIM configurations](../../../user/group/saml_sso/example_saml_config.md). +are not accepted as a source of groups. + +For more information on configuring the +required group attribute name in the SAML identity provider's settings, see +example configurations for [Azure AD](../../../user/group/saml_sso/example_saml_config.md#group-sync) and [Okta](../../../user/group/saml_sso/example_saml_config.md#group-sync-1). ## Configure SAML Group Links @@ -135,7 +137,7 @@ To integrate Microsoft Azure AD, you: <!-- vale gitlab.SentenceSpacing = NO --> -1. In the [Azure Portal](https://portal.azure.com), go to **Azure Active Directory > App registrations > All applications**, and select your GitLab SAML application. +1. In the [Azure Portal](https://portal.azure.com), go to **Microsoft Entra ID > App registrations > All applications**, and select your GitLab SAML application. 1. Under **Essentials**, the **Application (client) ID** and **Directory (tenant) ID** values are displayed. Copy these values, because you need them for the GitLab configuration. 1. In the left navigation, select **Certificates & secrets**. 1. On the **Client secrets** tab, select **New client secret**. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 734679cf331..444afd3442b 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -240,8 +240,7 @@ If you are having issues configuring GitLab, see the [troubleshooting documentat ## User access and management -> - SAML user provisioning [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/268142) in GitLab 13.7. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325712) in GitLab 14.0, GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning are displayed with an ][**Enterprise**](../../enterprise_user/index.md) badge in the **Members** view. +> SAML user provisioning [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/268142) in GitLab 13.7. After group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, see [user access](scim_setup.md#user-access) on the SCIM page. @@ -357,7 +356,7 @@ when the account was created either: #### Supported user attributes - **can_create_group** - `true` or `false` to indicate whether the user can create - new groups. Default is `true`. + new top-level groups. Default is `true`. - **projects_limit** - The total number of personal projects a user can create. A value of `0` means the user cannot create new projects in their personal namespace. Default is `10000`. @@ -532,6 +531,7 @@ immediately. If the user: - [SAML SSO for self-managed GitLab instances](../../../integration/saml.md) - [Glossary](../../../integration/saml.md#glossary) +- [Blog post: The ultimate guide to enabling SAML and SSO on GitLab.com](https://about.gitlab.com/blog/2023/09/14/the-ultimate-guide-to-enabling-saml/) - [Authentication comparison between SaaS and self-managed](../../../administration/auth/index.md#saas-vs-self-managed-comparison) - [Passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) - [SAML Group Sync](group_sync.md) diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index a9b9bf26444..e29b33469ab 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -118,8 +118,8 @@ For each attribute: 1. Select **Ok**. If your SAML configuration differs from [the recommended SAML settings](index.md#azure), select the mapping -attributes and modify them accordingly. In particular, the `objectId` source attribute must map to the `externalId` -target attribute. +attributes and modify them accordingly. The source attribute that you map to the `externalId` +target attribute must match the attribute used for the SAML `NameID`. If a mapping is not listed in the table, use the Azure Active Directory defaults. For a list of required attributes, refer to the [internal group SCIM API](../../../development/internal_api/index.md#group-scim-api) documentation. @@ -162,8 +162,6 @@ To configure Okta for SCIM: ## User access -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325712) in GitLab 14.0, GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning are displayed with an [**Enterprise**](../../enterprise_user/index.md) badge in the **Members** view. - During the synchronization process, all new users: - Receive GitLab accounts. diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md index cf9b9f5d4eb..9d3cc0bef50 100644 --- a/doc/user/group/saml_sso/troubleshooting.md +++ b/doc/user/group/saml_sso/troubleshooting.md @@ -1,7 +1,7 @@ --- type: reference stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -259,6 +259,14 @@ If you receive a `404` during setup when using "verify configuration", make sure If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#set-up-your-identity-provider), they may see a 404. As outlined in the [user access section](index.md#link-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. +If the top-level group has [restricted membership by email domain](../access_and_permissions.md#restrict-group-access-by-domain), and a user with an email domain that is not allowed tries to sign in with SSO, that user might receive a 404. Users might have multiple accounts, and their SAML identity might be linked to their personal account which has an email address that is different than the company domain. To check this, verify the following: + +- That the top-level group has restricted membership by email domain. +- That, in [Audit Events](../../../administration/audit_events.md) for the top-level group: + - You can see **Signed in with GROUP_SAML authentication** action for that user. + - That the user's username is the same as the username you configured for SAML SSO, by selecting the **Author** name. + - If the username is different to the username you configured for SAML SSO, ask the user to [unlink the SAML identity](index.md#unlink-accounts) from their personal account. + If all users are receiving a `404` after signing in to the identity provider (IdP): - Verify the `assertion_consumer_service_url`: diff --git a/doc/user/group/saml_sso/troubleshooting_scim.md b/doc/user/group/saml_sso/troubleshooting_scim.md index 63d10f3e932..703dff16fd5 100644 --- a/doc/user/group/saml_sso/troubleshooting_scim.md +++ b/doc/user/group/saml_sso/troubleshooting_scim.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index 795967e9f91..cd3efcc6562 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, howto --- @@ -138,7 +138,8 @@ token.revoke! ## Scopes for a group access token -> `k8s_proxy` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422408) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `k8s_proxy_pat`. Enabled by default. +> - `k8s_proxy` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422408) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `k8s_proxy_pat`. Enabled by default. +> - Feature flag `k8s_proxy_pat` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/131518) in GitLab 16.5. The scope determines the actions you can perform when you authenticate with a group access token. diff --git a/doc/user/group/troubleshooting.md b/doc/user/group/troubleshooting.md index 6de90053c21..08343f604f1 100644 --- a/doc/user/group/troubleshooting.md +++ b/doc/user/group/troubleshooting.md @@ -99,4 +99,4 @@ This error typically occurs when the user you're trying to remove is part of an - Remove the invited group membership from your project or group members page. - Recommended. Remove the user directly from the invited group, if you have access to the group. -The feature request to **Update billable_members endpoint to include invited group** is currently being worked on. For more information, see [issue 386583](https://gitlab.com/gitlab-org/gitlab/-/issues/386583) +The feature request to **Update `billable_members` endpoint to include invited group** is currently being worked on. For more information, see [issue 386583](https://gitlab.com/gitlab-org/gitlab/-/issues/386583) diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index 0d91416dfa5..df9986e32e7 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -204,6 +204,10 @@ You can change the name of a project environment in your GitLab CI/CD configurat > - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 13.3 > - Horizontal stage path [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in 13.0 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in 13.12 +> - Predefined date ranges dropdown list [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408656/) in GitLab 16.5 [with a flag](../../../administration/feature_flags.md) named `vsa_predefined_date_ranges`. Disabled by default. + +FLAG: +On self-managed GitLab, by default the predefined date ranges dropdown list feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `vsa_predefined_date_ranges`. On GitLab.com, this feature is not available. The feature is not ready for production use. Prerequisites: @@ -219,10 +223,10 @@ To view value stream analytics for your group or project: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. The charts and list show workflow items created - during the date range. + 1. To view metrics in a particular date range, from the dropdown list select a predefined date range or the **Custom** option. With the **Custom** option selected: + - In the **From** field, select a start date. + - In the **To** field, select an end date. + The charts and list display workflow items created during the date range. 1. Optional. Sort results by ascending or descending: - To sort by most recent or oldest workflow item, select the **Last event** header. - To sort by most or least amount of time spent in each stage, select the **Duration** header. diff --git a/doc/user/img/enable_AI_ML_features.png b/doc/user/img/enable_AI_ML_features.png Binary files differindex 577fb367e4d..97d06c877e4 100644 --- a/doc/user/img/enable_AI_ML_features.png +++ b/doc/user/img/enable_AI_ML_features.png diff --git a/doc/user/img/forecast_deployment_frequency.png b/doc/user/img/forecast_deployment_frequency.png Binary files differindex 4c8c6f47a08..7df7f46db3b 100644 --- a/doc/user/img/forecast_deployment_frequency.png +++ b/doc/user/img/forecast_deployment_frequency.png diff --git a/doc/user/img/linked_items_list_v16_5.png b/doc/user/img/linked_items_list_v16_5.png Binary files differnew file mode 100644 index 00000000000..29923e53276 --- /dev/null +++ b/doc/user/img/linked_items_list_v16_5.png diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 25605cdac62..1e6c59c2253 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -35,7 +35,7 @@ All templates: - Use the [GitLab-managed Terraform state](terraform_state.md) as the Terraform state storage backend. - Trigger four pipeline stages: `test`, `validate`, `build`, and `deploy`. - Run Terraform commands: `test`, `validate`, `plan`, and `plan-json`. It also runs the `apply` only on the default branch. -- Check for security problems using [IaC Scanning](../../application_security/iac_scanning/index.md#configure-iac-scanning-manually). +- Check for security problems using [IaC Scanning](../../application_security/iac_scanning/index.md). ### Latest Terraform template diff --git a/doc/user/markdown.md b/doc/user/markdown.md index c996b29c9a2..7f097891e92 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -262,7 +262,7 @@ this font installed by default. <!-- vale gitlab.Spelling = YES --> -To learn more about adding custom emoji, see [Custom emoji](award_emojis.md#custom-emoji). +To learn more about adding custom emoji, see [Custom emoji](emoji_reactions.md#custom-emoji). ### Front matter diff --git a/doc/user/okrs.md b/doc/user/okrs.md index bb58dcf516b..46390cd0275 100644 --- a/doc/user/okrs.md +++ b/doc/user/okrs.md @@ -354,6 +354,51 @@ Prerequisites: By default, child OKRs are ordered by creation date. To reorder them, drag them around. +### Schedule OKR check-in reminders + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422761) in GitLab 16.4 [with a flag](../administration/feature_flags.md) named `okr_checkin_reminders`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `okr_checkin_reminders`. +On GitLab.com, this feature is not available. +The feature is not ready for production use. + +Schedule check-in reminders to remind your team to provide status updates on the key results you care +about. +Reminders are sent to all assignees of descendant objects and key results as email notifications +and to-do items. +Users can't unsubscribe from the email notifications, but check-in reminders can be turned off. +Reminders are sent on Tuesdays. + +Prerequisites: + +- You must have at least the Reporter role for the project. +- There must be at least one objective with at least one key result in the project. +- You can schedule reminders only for top-level objectives. + Scheduling a check-in reminder for child objectives has no effect. + The setting from the top-level objective is inherited to all child objectives. + +To schedule a recurring reminder for an objective, in a new comment use the `/checkin_reminder <cadence>` +[quick action](project/quick_actions.md#work-items). +The options for `<cadence>` are: + +- `weekly` +- `twice-monthly` +- `monthly` +- `never` (default) + +For example, to schedule a weekly check-in reminder, enter: + +```plaintext +/checkin_reminder weekly +``` + +To turn off a check-in reminder, enter: + +```plaintext +/checkin_reminder never +``` + ## Confidential OKRs > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8410) in GitLab 15.3. @@ -442,3 +487,50 @@ The description and threads are on the left, and attributes, such as labels or assignees, on the right. ![OKR two column view](img/objective_two_column_view_v16_2.png) + +## Linked items in OKRs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/416558) in GitLab 16.5 [with a flag](../administration/feature_flags.md) named `linked_work_items`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `linked_work_items`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +Linked items are a bi-directional relationship and appear in a block below +the Child objectives and key results. You can link an objective, key result, or a task in the same project with each other. + +The relationship only shows up in the UI if the user can see both items. + +### Add a linked item + +Prerequisite: + +- You must have at least the Guest role for the project. + +To link an item to an objective or key result: + +1. In the **Linked items** section of an objective or key result, + select the **Add** button. +1. Select the relationship between the two items. Either: + - **relates to** + - **blocks** + - **is blocked by** +1. Enter the search text of the item. +1. When you have added all the items to be linked, select **Add** below the search box. + +When you have finished adding all linked items, you can see +them categorized so their relationships can be better understood visually. + +![Linked items block](img/linked_items_list_v16_5.png) + +### Remove a linked item + +Prerequisite: + +- You must have at least the Guest role for the project. + +In the **Linked items** section of an objective or key result, +next to each item, select the vertical ellipsis (**{ellipsis_v}**) and then select **Remove**. + +Due to the bi-directional relationship, the relationship no longer appears in either item. diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 3c80e739465..d8662ef6512 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -134,6 +134,7 @@ Prerequisites: with the scope set to, at minimum, `api`. - A [deploy token](../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. + - A [CI/CD Job token](../../../ci/jobs/ci_job_token.md) To install a package: @@ -221,6 +222,26 @@ To install a package: } ``` + Using a CI/CD job token: + + ```shell + composer config gitlab-token.<DOMAIN-NAME> gitlab-ci-token ${CI_JOB_TOKEN} + ``` + + Result in the `auth.json` file: + + ```json + { + ... + "gitlab-token": { + "<DOMAIN-NAME>": { + "username": "gitlab-ci-token", + "token": "<ci-job-token>", + ... + } + } + ``` + You can unset this with the command: ```shell @@ -312,11 +333,13 @@ You can install from source by pulling the Git repository directly. To do so, ei #### SSH access -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119739) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `composer_use_ssh_source_urls`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119739) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `composer_use_ssh_source_urls`. Disabled by default. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/329246) GitLab 16.5. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, an administrator can -[enable the feature flag](../../../administration/feature_flags.md) named `composer_use_ssh_source_urls`. +On self-managed GitLab, by default this feature is available. To hide the feature per project, an administrator can +[disable the feature flag](../../../administration/feature_flags.md) named `composer_use_ssh_source_urls`. +On GitLab.com, this feature is available. When you install from source, the `composer` configures an access to the project's Git repository. diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md index 8e555204f80..45ebfb2ef73 100644 --- a/doc/user/packages/debian_repository/index.md +++ b/doc/user/packages/debian_repository/index.md @@ -4,7 +4,7 @@ group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Debian packages in the Package Registry **(FREE ALL EXPERIMENT)** +# Debian packages in the Package Registry **(FREE SELF EXPERIMENT)** > - Debian API [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42670) in GitLab 13.5. > - Debian group API [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66188) in GitLab 14.2. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 13d84fa3d99..6765aa2cbb1 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -317,6 +317,10 @@ To publish a package by using Gradle: ## Publish a package +WARNING: +Using the `DeployAtEnd` option can cause an upload to be rejected with `400 bad request {"message":"Validation failed: Name has already been taken"}`. For more details, +see [issue 424238](https://gitlab.com/gitlab-org/gitlab/-/issues/424238). + After you have set up the [authentication](#authenticate-to-the-package-registry) and [chosen an endpoint for publishing](#naming-convention), publish a Maven package to your project. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 695193f878a..9d789c27d1f 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -83,23 +83,11 @@ NPM_TOKEN=your_token npm publish Your package should now publish to the Package Registry. -## Publishing a package via a CI/CD pipeline +## Publishing a package by using a CI/CD pipeline -### Authenticating via the `.npmrc` - -Create or edit the `.npmrc` file in the same directory as your `package.json` in a GitLab project. Include the following lines in the `.npmrc` file: +When publishing by using a CI/CD pipeline, you can use the [predefined variables](../../../ci/variables/predefined_variables.md) `${CI_PROJECT_ID}` and `${CI_JOB_TOKEN}` to authenticate with your project's Package Registry. We use these variables to create a `.npmrc` file [for authentication](#authenticating-via-the-npmrc) during execution of your CI/CD job. -```shell -@scope:registry=https://your_domain_name/api/v4/projects/your_project_id/packages/npm/ -//your_domain_name/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN} -``` - -- Replace `@scope` with the [root level group](#naming-convention) of the project you're publishing to the package to. -- The `${CI_PROJECT_ID}` and `${CI_JOB_TOKEN}` are [predefined variables](../../../ci/variables/predefined_variables.md) that are available in the pipeline and do not need to be replaced. - -### Publishing a package via a CI/CD pipeline - -In the GitLab project that houses your `.npmrc` and `package.json`, edit or create a `.gitlab-ci.yml` file. For example: +In the GitLab project containing your `package.json`, edit or create a `.gitlab-ci.yml` file. For example: ```yaml image: node:latest @@ -107,14 +95,17 @@ image: node:latest stages: - deploy -deploy: +publish-npm: stage: deploy script: - - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc + - echo "@scope:registry=https://${CI_SERVER_HOST}:${CI_SERVER_PORT}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/" > .npmrc + - echo "//${CI_SERVER_HOST}:${CI_SERVER_PORT}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}" >> .npmrc - npm publish ``` -Your package should now publish to the Package Registry when the pipeline runs. +- Replace `@scope` with the [scope](https://docs.npmjs.com/cli/v10/using-npm/scope) of the package that is being published. + +Your package is published to the Package Registry when the `publish-npm` job in your pipeline runs. ## Install a package diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 340df4a3c5f..f5430c5328c 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -492,6 +492,85 @@ dotnet add package <package_id> \ - `<package_id>` is the package ID. - `<package_version>` is the package version. Optional. +### Install a package using NuGet v2 feed + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/416405) in GitLab 16.5. + +Prerequisites: + +- The project-level Package Registry is a [v2 feed source](#add-a-source-with-chocolatey-cli) for Chocolatey. +- A version must be provided when installing or upgrading a package using NuGet v2 feed. + +To install a package with the Chocolatey CLI: + +```shell +choco install <package_id> -Source <source_url> -Version <package_version> +``` + +In this command: + +- `<package_id>` is the package ID. +- `<source_url>` is the URL or name of the NuGet v2 feed Package Registry. +- `<package_version>` is the package version. + +For example: + +```shell +choco install MyPackage -Source gitlab -Version 1.0.2 + +# or + +choco install MyPackage -Source "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/nuget/v2" -u <username> -p <gitlab_personal_access_token, deploy_token or job token> -Version 1.0.2 +``` + +To upgrade a package with the Chocolatey CLI: + +```shell +choco upgrade <package_id> -Source <source_url> -Version <package_version> +``` + +In this command: + +- `<package_id>` is the package ID. +- `<source_url>` is the URL or name of the NuGet v2 feed Package Registry. +- `<package_version>` is the package version. + +For example: + +```shell +choco upgrade MyPackage -Source gitlab -Version 1.0.3 +``` + +## Delete a package + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38275) in GitLab 16.5. + +WARNING: +Deleting a package is a permanent action that cannot be undone. + +Prerequisites: + +- You must have the [Maintainer](../../../user/permissions.md#project-members-permissions) role or higher in the project. +- You must have both the package name and version. + +To delete a package with the NuGet CLI: + +```shell +nuget delete <package_id> <package_version> -Source <source_name> -ApiKey <gitlab_personal_access_token, deploy_token or job token> +``` + +In this command: + +- `<package_id>` is the package ID. +- `<package_version>` is the package version. +- `<source_name>` is the source name. + +For example: + +```shell +nuget delete MyPackage 1.0.0 -Source gitlab -ApiKey <gitlab_personal_access_token, deploy_token or job token> +``` + ## Symbol packages > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/262081) in GitLab 14.1. @@ -512,6 +591,8 @@ for further updates. ## Supported CLI commands +> `nuget delete` and `dotnet nuget delete` commands [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38275) in GitLab 16.5. + The GitLab NuGet repository supports the following commands for the NuGet CLI (`nuget`) and the .NET CLI (`dotnet`): @@ -519,6 +600,8 @@ CLI (`dotnet`): - `dotnet nuget push`: Upload a package to the registry. - `nuget install`: Install a package from the registry. - `dotnet add`: Install a package from the registry. +- `nuget delete`: Delete a package from the registry. +- `dotnet nuget delete`: Delete a package from the registry. ## Example project diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 59184b811d4..e9c019deefa 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -89,7 +89,7 @@ For more information about using the GitLab Package Registry with CI/CD, see: - [Conan](../conan_repository/index.md#publish-a-conan-package-by-using-cicd) - [Generic](../generic_packages/index.md#publish-a-generic-package-by-using-cicd) - [Maven](../maven_repository/index.md#create-maven-packages-with-gitlab-cicd) -- [npm](../npm_registry/index.md#publishing-a-package-via-a-cicd-pipeline) +- [npm](../npm_registry/index.md#publishing-a-package-by-using-a-cicd-pipeline) - [NuGet](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) - [PyPI](../pypi_repository/index.md#authenticate-with-a-ci-job-token) - [RubyGems](../rubygems_registry/index.md#authenticate-with-a-ci-job-token) diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index cb0516bdc4a..5c4105f8e00 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -204,7 +204,7 @@ For example, if: - The project is `gitlab.example.com/parent-group/sub-group/my-project`. - The Terraform module is `my-infra-package`. -The project name must be unique in all projects in all groups under `parent-group`. +The module name must be unique in all projects in all groups under `parent-group`. ## Delete a Terraform module diff --git a/doc/user/packages/yarn_repository/index.md b/doc/user/packages/yarn_repository/index.md index 52accfc4fae..942e62ae3e7 100644 --- a/doc/user/packages/yarn_repository/index.md +++ b/doc/user/packages/yarn_repository/index.md @@ -76,10 +76,6 @@ You can use **Shared Runners** *(Default)* or **Private Runners** (Advanced). #### Shared runners -Third party images such as `node:latest` or `node:current` do not have direct access -to the `CI_JOB_TOKEN` when operating in a shared runner. You must configure an -authentication token or use a private runner. - To create an authentication token for your project or group: 1. On the left sidebar, select **Search or go to** and find your project or group. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index dadfb75ed4e..a83ce6a56c6 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -19,6 +19,7 @@ GitLab [administrators](../administration/index.md) have all permissions. The available roles are: - Guest (This role applies to [private and internal projects](../user/public_access.md) only.) +- [Custom](custom_roles.md) - Reporter - Developer - Maintainer @@ -247,7 +248,7 @@ The following table lists project permissions available for each role: 20. Maintainers cannot create, demote, or remove Owners, and they cannot promote users to the Owner role. They also cannot approve Owner role access requests. 21. Authors of tasks can delete them even if they don't have the Owner role, but they have to have at least the Guest role for the project. 22. You must have permission to [view the epic](group/epics/manage_epics.md#who-can-view-an-epic). -23. In GitLab 15.9 and later, users with the Guest role and an Ultimate license can view private repository content if an administrator (on self-managed) or group owner (on GitLab.com) gives those users permission. The administrator or group owner can create a [custom role](#custom-roles) through the API and assign that role to the users. +23. In GitLab 15.9 and later, users with the Guest role and an Ultimate license can view private repository content if an administrator (on self-managed) or group owner (on GitLab.com) gives those users permission. The administrator or group owner can create a [custom role](custom_roles.md) through the API and assign that role to the users. <!-- markdownlint-enable MD029 --> @@ -454,6 +455,7 @@ To work around the issue, give these users the Guest role or higher to any proje ## Related topics +- [Custom roles](custom_roles.md) - [The GitLab principles behind permissions](https://about.gitlab.com/handbook/product/gitlab-the-product/#permissions-in-gitlab) - [Members](project/members/index.md) - Customize permissions on [protected branches](project/protected_branches.md) @@ -465,182 +467,3 @@ To work around the issue, give these users the Guest role or higher to any proje - [Container Registry permissions](packages/container_registry/index.md#container-registry-visibility-permissions) - [Release permissions](project/releases/index.md#release-permissions) - [Read-only namespaces](../user/read_only_namespaces.md) - -## Custom roles **(ULTIMATE ALL)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106256) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `customizable_roles`. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110810) in GitLab 15.9. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114524) in GitLab 15.10. -> - The ability for a custom role to view a vulnerability report [introduced](https://gitlab.com/groups/gitlab-org/-/epics/10160) in GitLab 16.1 [with a flag](../administration/feature_flags.md) named `custom_roles_vulnerability`. -> - Ability to view a vulnerability report [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123835) in GitLab 16.1. -> - [Feature flag `custom_roles_vulnerability` removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124049) in GitLab 16.2. -> - Ability to create and remove a custom role with the UI [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393235) in GitLab 16.4. - -Custom roles allow group members who are assigned the Owner role to create roles -specific to the needs of their organization. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a demo of the custom roles feature, see [[Demo] Ultimate Guest can view code on private repositories via custom role](https://www.youtube.com/watch?v=46cp_-Rtxps). - -The following custom roles are available: - -- The Guest+1 role, which allows users with the Guest role to view code. -- In GitLab 16.1 and later, you can create a custom role that can view vulnerability reports and change the status of the vulnerabilities. -- In GitLab 16.3 and later, you can create a custom role that can view the dependency list. -- In GitLab 16.4 and later, you can create a custom role that can approve merge requests. - -You can discuss individual custom role and permission requests in [issue 391760](https://gitlab.com/gitlab-org/gitlab/-/issues/391760). - -When you enable a custom role for a user with the Guest role, that user has -access to elevated permissions, and therefore: - -- Is considered a [billable user](../subscriptions/self_managed/index.md#billable-users) on self-managed GitLab. -- [Uses a seat](../subscriptions/gitlab_com/index.md#how-seat-usage-is-determined) on GitLab.com. - -This does not apply to Guest+1, a Guest custom role that only enables the `read_code` -permission. Users with that specific custom role are not considered billable users -and do not use a seat. - -### Create a custom role - -Prerequisites: - -- You must be an administrator for the self-managed instance, or have the Owner - role in the group you are creating the custom role in. -- The group must be in the Ultimate tier. -- You must have: - - At least one private project so that you can see the effect of giving a - user with the Guest role a custom role. The project can be in the group itself - or one of that group's subgroups. - - A [personal access token with the API scope](profile/personal_access_tokens.md#create-a-personal-access-token). - -#### GitLab SaaS - -Prerequisite: - -- You must have the Owner role in the group you are creating the custom role in. - -1. On the left sidebar, select **Search or go to** and find your group. -1. Select **Settings > Roles and Permissions**. -1. Select **Add new role**. -1. In **Base role to use as template**, select **Guest**. -1. In **Role name**, enter the custom role's title. -1. Select the **Permissions** for the new custom role. -1. Select **Create new role**. - -#### Self Managed GitLab Instances - -Prerequisite: - -- You must be an administrator for the self-managed instance you are creating the custom role in. - -1. On the left sidebar, select **Search or go to**. -1. Select **Admin Area**. -1. Select **Settings > Roles and Permissions**. -1. From the top dropdown list, select the group you want to create a custom role in. -1. Select **Add new role**. -1. In **Base role to use as template**, select **Guest**. -1. In **Role name**, enter the custom role's title. -1. Select the **Permissions** for the new custom role. -1. Select **Create new role**. - -To create a custom role, you can also [use the API](../api/member_roles.md#add-a-member-role-to-a-group). - -#### Custom role requirements - -For every ability, a minimal access level is defined. To be able to create a custom role which enables a certain ability, the `member_roles` table record has to have the associated minimal access level. For all abilities, the minimal access level is Guest. Only users who have at least the Guest role can be assigned to a custom role. - -Some roles and abilities require having other abilities enabled. For example, a custom role can only have administration of vulnerabilities (`admin_vulnerability`) enabled if reading vulnerabilities (`read_vulnerability`) is also enabled. - -You can see the required minimal access levels and abilities requirements in the following table. - -| Ability | Minimal access level | Required ability | -| -- | -- | -- | -| `read_code` | Guest | - | -| `read_dependency` | Guest | - | -| `read_vulnerability` | Guest | - | -| `admin_merge_request` | Guest | - | -| `admin_vulnerability` | Guest | `read_vulnerability` | - -### Associate a custom role with an existing group member - -To associate a custom role with an existing group member, a group member with -the Owner role: - -1. Invites a user as a direct member to the root group or any subgroup or project in the root - group's hierarchy as a Guest. At this point, this Guest user cannot see any - code on the projects in the group or subgroup. -1. Optional. If the Owner does not know the `id` of the Guest user receiving a custom - role, finds that `id` by making an [API request](../api/member_roles.md#list-all-member-roles-of-a-group). - -1. Associates the member with the Guest+1 role using the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) - - ```shell - # to update a project membership - curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": '<member_role_id>', "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" - - # to update a group membership - curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": '<member_role_id>', "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" - ``` - - Where: - - - `<project_id` and `<group_id>`: The `id` or [URL-encoded path of the project or group](../api/rest/index.md#namespaced-path-encoding) associated with the membership receiving the custom role. - - `<member_role_id>`: The `id` of the member role created in the previous section. - - `<user_id>`: The `id` of the user receiving a custom role. - - Now the Guest+1 user can view code on all projects associated with this membership. - -### Remove a custom role - -Prerequisite: - -- You must be an administrator or have the Owner role in the group you are removing the custom role from. - -You can remove a custom role from a group only if no group members have that role. - -To do this, you can either remove the custom role from all group members with that custom role, or remove those members from the group. - -#### Remove a custom role from a group member - -To remove a custom role from a group member, use the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) -and pass an empty `member_role_id` value. - -```shell -# to update a project membership -curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/projects/<project_id>/members/<user_id>" - -# to update a group membership -curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer <your_access_token>" --data '{"member_role_id": "", "access_level": 10}' "https://gitlab.example.com/api/v4/groups/<group_id>/members/<user_id>" -``` - -#### Remove a group member with a custom role from the group - -1. On the left sidebar, select **Search or go to** and find your group. -1. Select **Manage > Members**. -1. On the member row you want to remove, select the vertical ellipsis - (**{ellipsis_v}**) and select **Remove member**. -1. In the **Remove member** confirmation dialog, do not select any checkboxes. -1. Select **Remove member**. - -#### Delete the custom role - -After you have made sure no group members have that custom role, delete the -custom role. - -1. On the left sidebar, select **Search or go to**. -1. GitLab.com only. Select **Admin Area**. -1. Select **Settings > Roles and Permissions**. -1. Select **Custom Roles**. -1. In the **Actions** column, select **Delete role** (**{remove}**) and confirm. - -To delete a custom role, you can also [use the API](../api/member_roles.md#remove-member-role-of-a-group). -To use the API, you must know the `id` of the custom role. If you do not know this -`id`, find it by making an [API request](../api/member_roles.md#list-all-member-roles-of-a-group). - -### Known issues - -- If a user with a custom role is shared with a group or project, their custom - role is not transferred over with them. The user has the regular Guest role in - the new group or project. -- You cannot use an [Auditor user](../administration/auditor_users.md) as a template for a custom role. diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md index d90b2d5c882..ca55ab758da 100644 --- a/doc/user/product_analytics/index.md +++ b/doc/user/product_analytics/index.md @@ -20,7 +20,11 @@ This feature is not ready for production use. This page is a work in progress, and we're updating the information as we add more features. For more information, see the [group direction page](https://about.gitlab.com/direction/analytics/product-analytics/). -To leave feedback about Product Analytics bugs or functionality, please comment in [issue 391970](https://gitlab.com/gitlab-org/gitlab/-/issues/391970) or open a new issue with the label `group::product analytics`. +To leave feedback about Product Analytics bugs or functionality: + +- Comment on [issue 391970](https://gitlab.com/gitlab-org/gitlab/-/issues/391970). +- Create an issue with the `group::product analytics` label. +- [Schedule a call](https://calendly.com/jheimbuck/30-minute-call) with the team. ## How product analytics works @@ -269,6 +273,19 @@ POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi If the request is successful, the returned JSON includes an array of rows of results. +## Onboarding GitLab internal projects + +GitLab team members can enable Product Analytics on their own internal projects on GitLab.com during the experiment phase. + +1. Send a message to the Product Analytics team (`#g_analyze_product_analytics`) informing them of the repository to be enabled. +1. Ensure that the project is within an Ultimate namespace. +1. Using ChatOps, enable both the `product_analytics_dashboards` and `combined_analytics_dashboards` + + ```plaintext + /chatops run feature set product_analytics_dashboards true --project=FULLPATH_TO_PROJECT + /chatops run feature set combined_analytics_dashboards true --project=FULLPATH_TO_PROJECT + ``` + ## Troubleshooting ### No events are collected diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md index c3612b787ac..2166b40b575 100644 --- a/doc/user/profile/account/create_accounts.md +++ b/doc/user/profile/account/create_accounts.md @@ -1,7 +1,7 @@ --- type: reference stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index cf6ee61660f..d41eee911f9 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -1,7 +1,7 @@ --- type: howto stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -61,13 +61,18 @@ When deleting users, you can either: - Delete the user and their contributions, including: - Abuse reports. - Emoji reactions. - - Epics. - Groups of which the user is the only user with the Owner role. + - Personal access tokens. + - Epics. - Issues. - Merge requests. - - Notes and comments. - - Personal access tokens. - Snippets. + - [Notes and comments](../../../api/notes.md) + on other users' [commits](../../project/repository/index.md#commit-changes-to-a-repository), + [epics](../../group/epics/index.md), + [issues](../../project/issues/index.md), + [merge requests](../../project/merge_requests/index.md) + and [snippets](../../snippets.md). An alternative to deleting is [blocking a user](../../../administration/moderate_users.md#block-a-user). diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index c7633b2c664..d1f1d28663e 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -167,7 +167,7 @@ On self-managed GitLab, by default this feature is available. On GitLab.com this You can use Cisco Duo as an OTP provider in GitLab. -DUO® is a registered trademark of Cisco Systems, Inc., and/or its affiliates in the United States and certain other countries. +DUO® is a registered trademark of Cisco Systems, Inc., and/or its affiliates in the United States and certain other countries. #### Prerequisites diff --git a/doc/user/profile/achievements.md b/doc/user/profile/achievements.md index 1b875e984a1..c208bd554bf 100644 --- a/doc/user/profile/achievements.md +++ b/doc/user/profile/achievements.md @@ -281,3 +281,24 @@ If you don't want to display achievements on your profile, you can opt out. To d 1. Select **Edit profile**. 1. In the **Main settings** section, clear the **Display achievements on your profile** checkbox. 1. Select **Update profile settings**. + +## Reorder achievements + +By default, achievements on your profile are displayed in ascending order by awarded date. + +To change the order of your achievements, call the [`userAchievementPrioritiesUpdate` GraphQL mutation](../../api/graphql/reference/index.md#mutationuserachievementprioritiesupdate) +with an ordered list of all prioritized achievements. + +```graphql +mutation { + userAchievementPrioritiesUpdate(input: { + userAchievementIds: ["gid://gitlab/Achievements::UserAchievement/<first user achievement id>", "gid://gitlab/Achievements::UserAchievement/<second user achievement id>"], + }) { + userAchievements { + id + priority + } + errors + } +} +``` diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index cff18654292..6536a992292 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -1,7 +1,7 @@ --- type: index, howto stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index c3361040a00..9135a142612 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -1,7 +1,7 @@ --- type: concepts, howto stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -106,6 +106,7 @@ To view the last time a token was used: > - Personal access tokens no longer being able to access container or package registries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387721) in GitLab 16.0. > - `k8s_proxy` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422408) in GitLab 16.4 [with a flag](../../administration/feature_flags.md) named `k8s_proxy_pat`. Enabled by default. +> - Feature flag `k8s_proxy_pat` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/131518) in GitLab 16.5. A personal access token can perform actions based on the assigned scopes. diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index f057e62694b..170545d851f 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -49,7 +49,7 @@ To view the updated syntax highlighting theme, refresh your project's page. To customize the syntax highlighting theme, you can also [use the Application settings API](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). Use `default_syntax_highlighting_theme` to change the syntax highlighting colors on a more granular level. If these steps do not work, your programming language might not be supported by the syntax highlighters. -For more information, view [Rouge Ruby Library](https://github.com/rouge-ruby/rouge) for guidance on code files and Snippets. View [Moncaco Editor](https://microsoft.github.io/monaco-editor/) and [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) for guidance on the Web IDE. +For more information, view [Rouge Ruby Library](https://github.com/rouge-ruby/rouge) for guidance on code files and Snippets. View [Monaco Editor](https://microsoft.github.io/monaco-editor/) and [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) for guidance on the Web IDE. ## Change the diff colors @@ -105,17 +105,17 @@ To change the default content on your group overview page: 1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. Go to the **Behavior** section. -1. For **Group overivew content**, select an option. +1. For **Group overview content**, select an option. 1. Select **Save changes**. ### Customize default content on your project overview page -Your project overview page is the page you view when you select **Project overview** on the left sidebar. You can set your main project overview page to the Activity page, the Readme file, and other content. +Your project overview page is the page you view when you select **Project overview** on the left sidebar. You can set your main project overview page to the Activity page, the README file, and other content. 1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. Go to the **Behavior** section. -1. For **Project overivew content**, select an option. +1. For **Project overview content**, select an option. 1. Select **Save changes**. ### Hide shortcut buttons @@ -260,7 +260,7 @@ Customize the format used to display times of activities on your group and proje - Relative format, for example `30 minutes ago`. - Absolute format, for example `September 3, 2022, 3:57 PM`. -To use relative times on the GitLab UI: +To use exact times on the GitLab UI: 1. On the left sidebar, select your avatar. 1. Select **Preferences**. @@ -303,7 +303,9 @@ To access your **Followers** and **Following** tabs: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121079) in GitLab 16.1 as [Beta](../../policy/experiment-beta-support.md#beta). -To enable [Code Suggestions](../../user/project/repository/code_suggestions/index.md): +Code Suggestions are disabled by default at the user account level. + +To update this setting: 1. On the left sidebar, select your avatar. 1. Select **Preferences**. @@ -311,7 +313,7 @@ To enable [Code Suggestions](../../user/project/repository/code_suggestions/inde 1. Select **Save changes**. NOTE: -If Code Suggestions are turned off [for the group](../../user/group/manage.md#enable-code-suggestions), then you cannot enable them for yourself. (Your setting has no effect.) +If Code Suggestions are disabled [for any groups that you belong to](../../user/group/manage.md#enable-code-suggestions), then you cannot enable them for yourself. (Your setting has no effect.) ## Integrate your GitLab instance with third-party services diff --git a/doc/user/profile/service_accounts.md b/doc/user/profile/service_accounts.md index 8845ee55e14..6bb96b9c552 100644 --- a/doc/user/profile/service_accounts.md +++ b/doc/user/profile/service_accounts.md @@ -1,7 +1,7 @@ --- type: index, howto stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/profile/user_passwords.md b/doc/user/profile/user_passwords.md index 7882502a588..1b4fbd5fa53 100644 --- a/doc/user/profile/user_passwords.md +++ b/doc/user/profile/user_passwords.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index c0a50dade31..73d3d97be4a 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -51,15 +51,22 @@ GitLab authorizes the creator of the deploy key if the Git-command triggers addi ## Security implications The intended use case for deploy keys is for non-human interaction with GitLab, for example: an automated script running on a server in your organization. + +You should create a dedicated account to act as a service account, and create the deploy key with the service account. +If you use another user account to create deploy keys, the user is granted persistent privileges. + +In addition: + +- Deploy keys work even if the user who created them is removed from the group or project. +- The creator of a deploy key retains access to the group or project, even if the user is demoted or removed. +- When a deploy key is specified in a protected branch rule, the creator of the deploy key gains access to the protected branch, as well as to the deploy key itself. + As with all sensitive information, you should ensure only those who need access to the secret can read it. For human interactions, use credentials tied to users such as Personal Access Tokens. To help detect a potential secret leak, you can use the [Audit Event](../../../administration/audit_event_streaming/examples.md#example-payloads-for-ssh-events-with-deploy-key) feature. -WARNING: -Deploy keys work even if the user who created them is removed from the group or project. - ## View deploy keys To view the deploy keys available to a project: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 75d25e29bd9..4da756b05ea 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -32,9 +32,6 @@ When importing projects: creates comments describing that non-existent users were added as reviewers and approvers. However, the actual reviewer status and approval are not applied to the merge request in GitLab. - You can change the target namespace and target repository name before you import. -- The importer also imports branches on forks of projects related to open pull requests. These branches are - imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to - a discrepancy in branches compared to those of the GitHub repository. - The organization the repository belongs to must not impose restrictions of a [third-party application access policy](https://docs.github.com/en/organizations/managing-oauth-access-to-your-organizations-data/about-oauth-app-access-restrictions) on the GitLab instance you import to. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> @@ -161,14 +158,15 @@ When the **Organization** tab is selected, you can further narrow down your sear To make imports as fast as possible, the following items aren't imported from GitHub by default: - Issue and pull request events. For example, _opened_ or _closed_, _renamed_, and _labeled_ or _unlabeled_. -- All comments. In regular import of large repositories some comments might get skipped due to limitation of GitHub API. +- More than approximately 30,000 comments because of a [limitation of the GitHub API](#missing-comments). - Markdown attachments from repository comments, release posts, issue descriptions, and pull request descriptions. These can include images, text, or binary attachments. If not imported, links in Markdown to attachments break after you remove the attachments from GitHub. You can choose to import these items, but this could significantly increase import time. To import these items, select the appropriate fields in the UI: - **Import issue and pull request events**. -- **Use alternative comments import method**. +- **Use alternative comments import method**. If importing GitHub projects with more than approximately 30,000 comments, you should enable this method because of a + [limitation of the GitHub API](#missing-comments). - **Import Markdown attachments**. - **Import collaborators** (selected by default). Leaving it selected might result in new users using a seat in the group or namespace, and being granted permissions [as high as project owner](#collaborators-members). Only direct collaborators are imported. @@ -250,6 +248,8 @@ The following items of a project are imported: - Repository description. - Git repository data. +- All project branches. +- All branches of forks of the project related to open pull requests, but not closed pull requests. Branches from forks are imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. - Branch protection rules. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22650) in GitLab 15.4. - Collaborators (members). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388716) in GitLab 15.10. From GitLab 16.0, can be imported [as an additional item](#select-additional-items-to-import). @@ -475,19 +475,20 @@ repository to be imported manually. Administrators can manually import the repos The GitHub importer might encounter some errors when importing large projects. -#### Alternative way to import notes and diff notes +#### Missing comments -When the GitHub importer runs on extremely large projects, not all notes and diff notes can be imported due to the GitHub API `issues_comments` and `pull_requests_comments` endpoint limitations. - -If it's not possible to fetch all pages, the GitHub API might return the following error: +The GitHub API has a limit that prevents more than approximately 30,000 notes or diff notes from being imported. +When this limit is reached, the GitHub API instead returns the following error: ```plaintext In order to keep the API fast for everyone, pagination is limited for this resource. Check the rel=last link relation in the Link response header to see how far back you can traverse. ``` -An [alternative approach](#select-additional-items-to-import) for importing comments is available. +For example, see [this GitHub API response](https://api.github.com/repositories/27193779/issues/comments?page=401&per_page=100). -Instead of using `issues_comments` and `pull_requests_comments`, use individual resources to pull notes from one object at a time. This way, you can carry over any missing comments. However, execution takes longer because this method increases the number of network requests required to perform the import. +If you are importing GitHub projects with a large number of comments, you should select the **Use alternative comments import method** +[additional item to import](#select-additional-items-to-import) checkbox. This setting makes the import process take longer because it increases the number of network requests +required to perform the import. #### Reduce GitHub API request objects per page diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index df831c2f3eb..e47e9715980 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -48,6 +48,8 @@ GitLab self-managed administrators can reduce their attack surface by disabling 1. Scroll to **Import sources**. 1. Clear checkboxes for importers that are not required. +In GitLab 16.1 and earlier, you should **not** use direct transfer with [scheduled scan execution policies](../../../user/application_security/policies/scan-execution-policies.md). + ## Available project importers You can import projects from: diff --git a/doc/user/project/import/perforce.md b/doc/user/project/import/perforce.md index 86981799739..10e61139d50 100644 --- a/doc/user/project/import/perforce.md +++ b/doc/user/project/import/perforce.md @@ -56,7 +56,7 @@ Here's a few links to get you started: - [Git book migration guide](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git#_perforce_import) `git p4` and `git filter-branch` are not very good at -creating small and efficient Git pack files. So it might be a good +creating small and efficient Git packfiles. So it might be a good idea to spend time and CPU to properly repack your repository before sending it for the first time to your GitLab server. See [this StackOverflow question](https://stackoverflow.com/questions/28720151/git-gc-aggressive-vs-git-repack/). diff --git a/doc/user/project/integrations/aws_codepipeline.md b/doc/user/project/integrations/aws_codepipeline.md new file mode 100644 index 00000000000..b081544199e --- /dev/null +++ b/doc/user/project/integrations/aws_codepipeline.md @@ -0,0 +1,114 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# AWS CodePipeline **(FREE SAAS)** + +> [Introduced](https://gitlab.com/gitlab-com/alliances/aws/wip/aws-cs-collab/aws-gitlab-collaboration/-/issues/25) in GitLab 16.5. + +You can use your GitLab project to build, test, and deploy code changes using [AWS CodePipeline](https://aws.amazon.com/codepipeline/). To do so, you use: + +- AWS CodeStar Connections to connect your GitLab.com account to AWS. +- That connection to automatically start a pipeline based on changes to your code. + +## Create a connection from AWS CodePipeline to GitLab + +Prerequisites: + +- You must have the Owner role on the GitLab project that you are connecting with AWS CodePipeline. +- You must have the appropriate permissions to create a connection in AWS. +- You must use a supported AWS region. Unsupported regions (also listed in the [AWS documentation](https://docs.aws.amazon.com/codepipeline/latest/userguide/connections-gitlab.html)) are: + - Asia Pacific (Hong Kong). + - Africa (Cape Town). + - Middle East (Bahrain). + - Europe (Zurich). + - AWS GovCloud (US-West and US-East). + +To create a connection to a project on GitLab.com, you can use either the AWS Management Console, or the AWS Command Line Interface (AWS CLI). + +### Use the AWS Management Console + +To connect a new or existing pipeline in AWS CodePipeline with GitLab.com, first authorize the AWS connection to use your GitLab account. + +1. Sign in to the AWS Management Console, and open the [AWS Developer Tools console](https://console.aws.amazon.com/codesuite/settings/connections). +1. Select **Settings** > **Connections** > **Create connection**. +1. In **Select a provider**, select **GitLab**. +1. In **Connection name**, enter the name for the connection that you want to create and select **Connect to GitLab**. +1. In the GitLab sign-in page, enter your credentials and select **Sign in**. +1. An authorization page displays with a message requesting authorization for the connection to access your GitLab account. Select **Authorize**. +1. The browser returns to the connections console page. In the **Create GitLab connection** section, the new connection is shown in **Connection name**. +1. Select **Connect to GitLab**. After the connection is created successfully, a success banner displays. The connection details are shown on the **Connection settings** page. + +Now you've connected AWS CodeSuite to GitLab.com, you can create or edit a pipeline in AWS CodePipeline that leverages your GitLab projects. + +1. Sign in to the [AWS CodePipeline console](https://console.aws.amazon.com/codesuite/codepipeline/start). +1. Create or edit a pipeline: + - If you are creating a pipeline: + - Complete the fields in the first screen and select **Next**. + - On the **Source** page, in the **Source Provider** section, select **GitLab**. + - If you are editing an existing pipeline: + - Select **Edit** > **Edit stage** to add or edit your source action. + - On the **Edit action** page, in the **Action name** section, enter the name for your action. + - In **Action provider**, select **GitLab**. +1. In **Connection**, select the connection you created earlier. +1. In **Repository name**, to choose the name of your GitLab project, specify the full project path with the namespace and all subgroups. + For example, for a group-level project, enter the project name in the following format: `group-name/subgroup-name/project-name`. + The project path with the namespace is in the URL in GitLab. Do not copy URLs from the Web IDE or raw views as they contain other special URL segments. + You can also pick an option from the dialog, or type a new path manually. + For more information about the: + - Path and namespace, see the `path_with_namespace` field in the [projects API](../../../api/projects.md#get-single-project). + - Namespace in GitLab, see [namespaces](../../namespace/index.md). + +1. In **Branch name**, select the branch where you want your pipeline to detect source changes. + If the branch name does not populate automatically, this might be because of one of the following: + - You do not have the Owner role for the project. + - The project name is not valid. + - The connection used does not have access to the project. + +1. In **Output artifact format**, select the format for your artifacts. To store: + - Output artifacts from the GitLab action using the default method, select **CodePipeline default**. The action accesses the files from the GitLab repository and + stores the artifacts in a ZIP file in the pipeline artifact store. + - A JSON file that contains a URL reference to the repository so that downstream actions can perform Git commands directly, select **Full clone**. This option can only be used + by CodeBuild downstream actions. To choose this option: + - [Update the permissions for your CodeBuild project service role](https://docs.aws.amazon.com/codepipeline/latest/userguide/troubleshooting.html#codebuild-role-connections). + - Follow the [AWS CodePipeline tutorial on how to use full clone with a GitHub pipeline source](https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-github-gitclone.html). +1. Save the source action and continue. + +### Use the AWS CLI + +To use the AWS CLI to create a connection: + +- Use the `create-connection` command. +- Go to the AWS Console to authenticate with your GitLab.com account. +- Connect your GitLab project to AWS CodePipeline. + +To use the `create-connection` command: + +1. Open a terminal (Linux, macOS, or Unix) or command prompt (Windows). Use the AWS CLI to run the `create-connection` command, + specifying the `--provider-type` and `--connection-name` for your connection. In this example, the third-party provider name is + `GitLab` and the specified connection name is `MyConnection`. + + ```shell + aws codestar-connections create-connection --provider-type GitLab --connection-name MyConnection + ``` + + If successful, this command returns the connection's Amazon Resource Name (ARN) information. For example: + + ```json + { + "ConnectionArn": "arn:aws:codestar-connections:us-west-2:account_id:connection/aEXAMPLE-8aad-4d5d-8878-dfcab0bc441f" + } + ``` + +1. The new connection is created with a `PENDING` status by default. Use the console to change the connection's status to `AVAILABLE`. + +1. [Use the AWS Console to complete the connection](#use-the-aws-management-console). Make sure you select your pending GitLab connection. Do not select **Create connection**. + +## Related topics + +- [Announcement that AWS CodePipeline supports GitLab](https://aws.amazon.com/about-aws/whats-new/2023/08/aws-codepipeline-supports-gitlab/) +- [GitLab connections - AWS CodePipeline](https://docs.aws.amazon.com/codepipeline/latest/userguide/connections-gitlab.html) +- [Create a connection to GitLab - Developer Tools console](https://docs.aws.amazon.com/dtconsole/latest/userguide/connections-create-gitlab.html) +- [CodeStarSourceConnection for Bitbucket, GitHub, GitHub Enterprise Server, and GitLab actions - AWS CodePipeline](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodestarConnectionSource.html) diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 242d5e6974c..85cab40626d 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -41,7 +41,7 @@ integration in GitLab. 1. Select **Atlassian Bamboo**. 1. Ensure the **Active** checkbox is selected. 1. Enter the base URL of your Bamboo server. For example, `https://bamboo.example.com`. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#ssl-verification). 1. Enter the [build key](#identify-the-bamboo-build-plan-build-key) from your Bamboo build plan. 1. If necessary, enter a username and password for a Bamboo user that has @@ -124,7 +124,7 @@ For example: ### Builds not triggered If builds are not triggered, ensure you entered the right GitLab IP address in -Bamboo under **Trigger IP addresses**. Also check [integration webhook logs](index.md#troubleshooting) for request failures. +Bamboo under **Trigger IP addresses**. Also, check the integration webhook logs for request failures. ### Advanced Atlassian Bamboo features not available in GitLab UI diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index febbee66c33..8b04d6aad8e 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -58,4 +58,4 @@ internal issue tracker, the internal issue is linked. ## Troubleshooting -For recent integration webhook deliveries, check [integration webhook logs](index.md#troubleshooting). +For recent integration webhook deliveries, check the integration webhook logs. diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index d762c71242d..6f70305ce8b 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -168,19 +168,8 @@ The following events are available for Slack notifications: | **Wiki page** | A wiki page is created or updated. | | **Deployment** | A deployment starts or finishes. | | **Alert** | A new, unique alert is recorded. | -| **[Group mention](#trigger-notifications-for-group-mentions) in public** | A group is mentioned in a public context. | -| **[Group mention](#trigger-notifications-for-group-mentions) in private** | A group is mentioned in a confidential context. | | [**Vulnerability**](../../application_security/vulnerabilities/index.md) | A new, unique vulnerability is recorded. | -### Trigger notifications for group mentions - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/417751) in GitLab 16.4. - -To trigger a [notification event](#notification-events) for a group mention, use `@<group_name>` in: - -- Issue and merge request descriptions -- Comments on issues, merge requests, and commits - ## Troubleshooting ### GitLab for Slack app does not appear in the list of integrations diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 59b5043b8f7..01dbdd0b3f2 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -6,120 +6,181 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project integrations **(FREE ALL)** -You can integrate your GitLab projects with other applications. Integrations are -like plugins, and give you the freedom to add -functionality to GitLab. +NOTE: +This page contains information about configuring project integrations on GitLab.com. For administrator documentation, see [Project integration administration](../../../administration/settings/project_integration_management.md). + +You can integrate with external applications to add functionality to GitLab. + +You can view and manage integrations at the: + +- [Instance level](../../../administration/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration) (self-managed GitLab) +- [Group level](#manage-group-level-default-settings-for-a-project-integration) + +You can use: + +- [Instance-level or group-level default settings for a project integration](#use-instance-level-or-group-level-default-settings-for-a-project-integration) +- [Custom settings for a project or group integration](#use-custom-settings-for-a-project-or-group-integration) + +## Manage group-level default settings for a project integration + +Prerequisite: + +- You must have at least the Maintainer role for the group. + +To manage group-level default settings for a project integration: + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Integrations**. +1. Select an integration. +1. Complete the fields. +1. Select **Save changes**. + +WARNING: +This may affect all or most of the subgroups and projects belonging to the group. Review the details below. + +If this is the first time you are setting up group-level settings for an integration: + +- The integration is enabled for all subgroups and projects belonging to the group that don't already have + this integration configured, if you have the **Enable integration** toggle turned on in the group-level + settings. +- Subgroups and projects that already have the integration configured are not affected, but can choose to use + the inherited settings at any time. + +When you make further changes to the group defaults: + +- They are immediately applied to all subgroups and projects belonging to the group that have the integration + set to use default settings. +- They are immediately applied to newer subgroups and projects, even those created after you last saved defaults for the + integration. If your group-level default setting has the **Enable integration** toggle turned on, + the integration is automatically enabled for all such subgroups and projects. +- Subgroups and projects with custom settings selected for the integration are not immediately affected and + may choose to use the latest defaults at any time. + +If [instance-level settings](../../../administration/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration) +have also been configured for the same integration, projects in the group inherit settings from the group. + +Only the entire settings for an integration can be inherited. Per-field inheritance +is proposed in [epic 2137](https://gitlab.com/groups/gitlab-org/-/epics/2137). + +### Remove a group-level default setting -## View project integrations +Prerequisite: -Prerequisites: +- You must have at least the Maintainer role for the group. + +To remove a group-level default setting: + +1. On the left sidebar, select **Search or go to** and find your group. +1. Select **Settings > Integrations**. +1. Select an integration. +1. Select **Reset** and confirm. + +Resetting a group-level default setting removes integrations that use default settings and belong to a project or subgroup of the group. + +## Use instance-level or group-level default settings for a project integration + +Prerequisite: - You must have at least the Maintainer role for the project. -To view the available integrations for your project: +To use instance-level or group-level default settings for a project integration: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Settings > Integrations**. +1. Select an integration. +1. On the right, from the dropdown list, select **Use default settings**. +1. Under **Enable integration**, ensure the **Active** checkbox is selected. +1. Complete the fields. +1. Select **Save changes**. -You can also view and manage integration settings across [all projects in an instance or group](../../admin_area/settings/project_integration_management.md). -For a single project, you can choose to inherit the instance or group configuration, -or provide custom settings. +## Use custom settings for a project or group integration -NOTE: -Instance and group-based integration management replaces service templates, which -were [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268032) in GitLab 14.0. +Prerequisite: -## Manage SSL verification +- You must have at least the Maintainer role for the project or group. -By default, the SSL certificate for outgoing HTTP requests is verified based on -an internal list of Certificate Authorities. This means the certificate cannot -be self-signed. +To use custom settings for a project or group integration: -You can turn off SSL verification in the configuration settings for [webhooks](webhooks.md#configure-a-webhook-in-gitlab) -and some integrations. +1. On the left sidebar, select **Search or go to** and find your project or group. +1. Select **Settings > Integrations**. +1. Select an integration. +1. On the right, from the dropdown list, select **Use custom settings**. +1. Under **Enable integration**, ensure the **Active** checkbox is selected. +1. Complete the fields. +1. Select **Save changes**. ## Available integrations -You can configure the following integrations. - | Integration | Description | Integration hooks | |-----------------------------------------------------------------------------|-----------------------------------------------------------------------|------------------------| | [Asana](asana.md) | Add commit messages as comments to Asana tasks. | **{dotted-circle}** No | -| Assembla | Manage projects. | **{dotted-circle}** No | -| [Atlassian Bamboo CI](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | -| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No | +| Assembla | Manage projects with Assembla. | **{dotted-circle}** No | +| [Atlassian Bamboo](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | +| [Bugzilla](bugzilla.md) | Use Bugzilla as an issue tracker. | **{dotted-circle}** No | | Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes | -| Campfire | Connect to chat. | **{dotted-circle}** No | -| [ClickUp](clickup.md) | Use ClickUp as the issue tracker. | **{dotted-circle}** No | -| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Use Confluence Cloud Workspace as an internal wiki. | **{dotted-circle}** No | +| Campfire | Connect Campfire to chat. | **{dotted-circle}** No | +| [ClickUp](clickup.md) | Use ClickUp as an issue tracker. | **{dotted-circle}** No | +| [Confluence Workspace](../../../api/integrations.md#confluence-workspace) | Use Confluence Workspace as an internal wiki. | **{dotted-circle}** No | | [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | | [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | | [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | -| Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | -| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | -| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No | +| Drone | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | +| [Emails on push](emails_on_push.md) | Send commits and diffs on push by email. | **{dotted-circle}** No | +| [Engineering Workflow Management (EWM)](ewm.md) | Use EWM as an issue tracker. | **{dotted-circle}** No | | [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No | -| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | +| [GitHub](github.md) | Receive statuses for commits and pull requests. | **{dotted-circle}** No | +| [GitLab for Slack app](gitlab_slack_application.md) | Use the native Slack app to receive notifications and run commands. | **{dotted-circle}** No | | [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat. | **{dotted-circle}** No | -| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | +| [Harbor](harbor.md) | Use Harbor as the Container Registry for GitLab. | **{dotted-circle}** No | | [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | | [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | -| JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | -| [Jira](../../../integration/jira/index.md) | Use Jira as the issue tracker. | **{dotted-circle}** No | +| JetBrains TeamCity | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | +| [Jira](../../../integration/jira/index.md) | Use Jira as an issue tracker. | **{dotted-circle}** No | | [Mattermost notifications](mattermost.md) | Send notifications about project events to Mattermost channels. | **{dotted-circle}** No | -| [Mattermost slash commands](mattermost_slash_commands.md) | Perform common tasks with slash commands. | **{dotted-circle}** No | -| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | -| Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | -| [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | +| [Mattermost slash commands](mattermost_slash_commands.md) | Run slash commands from a Mattermost chat environment. | **{dotted-circle}** No | +| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications in Microsoft Teams. | **{dotted-circle}** No | +| Packagist | Update your PHP dependencies in Packagist. | **{check-circle}** Yes | +| [Pipeline status emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | | [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | | [Pumble](pumble.md) | Send event notifications to a Pumble channel. | **{dotted-circle}** No | | Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | -| [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | -| [Shimo](shimo.md) (deprecated) | Use Shimo instead of the GitLab Wiki. | **{dotted-circle}** No | -| [GitLab for Slack app](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | -| [Slack notifications](slack.md) (deprecated) | Send notifications about project events to Slack. | **{dotted-circle}** No | -| [Slack slash commands](slack_slash_commands.md) | Enable slash commands in a workspace. | **{dotted-circle}** No | +| [Redmine](redmine.md) | Use Redmine as an issue tracker. | **{dotted-circle}** No | +| [Slack slash commands](slack_slash_commands.md) | Run slash commands from a Slack chat environment. | **{dotted-circle}** No | | [Squash TM](squash_tm.md) | Update Squash TM requirements when GitLab issues are modified. | **{check-circle}** Yes | | [Telegram](telegram.md) | Send notifications about project events to Telegram. | **{dotted-circle}** No | | [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | -| [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | -| [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | -| [ZenTao](zentao.md) (deprecated) | Use ZenTao as the issue tracker. | **{dotted-circle}** No | +| [Webex Teams](webex_teams.md) | Receive event notifications in Webex Teams. | **{dotted-circle}** No | +| [YouTrack](youtrack.md) | Use YouTrack as an issue tracker. | **{dotted-circle}** No | -### Project webhooks +## Project webhooks -You can configure a project webhook to listen for specific events -like pushes, issues, or merge requests. When the webhook is triggered, GitLab -sends a POST request with data to a specified webhook URL. +Some integrations use [webhooks](webhooks.md) for external applications. -For more information, see [Webhooks](webhooks.md). +You can configure a project webhook to listen for specific events +like pushes, issues, or merge requests. When the webhook is triggered, +GitLab sends a POST request with data to a specified webhook URL. -## Push hooks limit +For a list of integrations that use webhooks, see [Available integrations](#available-integrations). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17874) in GitLab 12.4. +## Push hook limit If a single push includes changes to more than three branches or tags, integrations -supported by `push_hooks` and `tag_push_hooks` events aren't executed. - -You can change the number of supported branches or tags by changing the -[`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). - -## Contribute to integrations +supported by `push_hooks` and `tag_push_hooks` events are not executed. -If you're interested in developing a new native integration for GitLab, see: +To change the number of supported branches or tags, configure the +[`push_event_hooks_limit` setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). -- [Integrations development guidelines](../../../development/integrations/index.md) -- [GitLab Developer Portal](https://developer.gitlab.com) - -## Troubleshooting +## SSL verification -Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). For more information, see [webhook troubleshooting](webhooks.md#troubleshooting). +By default, the SSL certificate for outgoing HTTP requests is verified based on +an internal list of certificate authorities. The SSL certificate cannot +be self-signed. -### `Test Failed. Save Anyway` error +You can disable SSL verification when you configure +[webhooks](webhooks.md#configure-a-webhook-in-gitlab) and some integrations. -Some integrations fail with an error `Test Failed. Save Anyway` when you set them -up on uninitialized repositories. This error occurs because the integration uses -push data to build the test payload, and there are no push events in the project. +## Related topics -To resolve this error, initialize the repository by pushing a test file to the project -and set up the integration again. +- [Integrations API](../../../api/integrations.md) +- [Integration development guidelines](../../../development/integrations/index.md) +- [GitLab Developer Portal](https://developer.gitlab.com) diff --git a/doc/user/project/integrations/mock_ci.md b/doc/user/project/integrations/mock_ci.md index fe702766a1d..ddd000f4c0e 100644 --- a/doc/user/project/integrations/mock_ci.md +++ b/doc/user/project/integrations/mock_ci.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Mock CI **(FREE ALL)** NOTE: -This integration only appears if you're in a [development environment](https://gitlab.com/gitlab-org/gitlab-mock-ci-service#setup-mockci-integration). +This integration is only available in a development environment. To set up the mock CI service server, respond to the following endpoints: @@ -17,4 +17,4 @@ To set up the mock CI service server, respond to the following endpoints: - `build_page`: `#{project.namespace.path}/#{project.path}/status/#{sha}` - Where the build is linked to (whether or not it's implemented). -For an example of a mock CI server, see [`gitlab-org/gitlab-mock-ci-service`](https://gitlab.com/gitlab-org/gitlab-mock-ci-service). +For an example Mock CI server, see [`gitlab-org/gitlab-mock-ci-service`](https://gitlab.com/gitlab-org/gitlab-mock-ci-service). diff --git a/doc/user/project/integrations/servicenow.md b/doc/user/project/integrations/servicenow.md index 2d02a4f631c..156ff4a54aa 100644 --- a/doc/user/project/integrations/servicenow.md +++ b/doc/user/project/integrations/servicenow.md @@ -17,7 +17,7 @@ With the GitLab spoke in ServiceNow, you can automate actions for GitLab projects, groups, users, issues, merge requests, branches, and repositories. For a full list of features, see the -[GitLab spoke documentation](https://docs.servicenow.com/bundle/sandiego-application-development/page/administer/integrationhub-store-spokes/concept/gitlab-spoke.html). +[GitLab spoke documentation](https://docs.servicenow.com/bundle/tokyo-application-development/page/administer/integrationhub-store-spokes/concept/gitlab-spoke.html). You must [configure GitLab as an OAuth2 authentication service provider](../../../integration/oauth_provider.md), which involves creating an application and then providing the Application ID diff --git a/doc/user/project/integrations/shimo.md b/doc/user/project/integrations/shimo.md index aeddee29109..6ea0cfa6fff 100644 --- a/doc/user/project/integrations/shimo.md +++ b/doc/user/project/integrations/shimo.md @@ -4,20 +4,17 @@ group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> - # Shimo (deprecated) **(FREE ALL)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 14.5 [with a flag](../../../administration/feature_flags.md) named `shimo_integration`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) in GitLab 15.4. Feature flag `shimo_integration` removed. + WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/377824) in GitLab 15.7 -and is planned for removal in 16.0. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/377824) in GitLab 15.7. This change is a breaking change. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) in GitLab 14.5 with a feature flag named `shimo_integration`. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) in GitLab 15.4. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) in GitLab 15.4. [Feature flag `shimo_integration`](https://gitlab.com/gitlab-org/gitlab/-/issues/345356) removed. - -[Shimo](https://shimo.im/) is a productivity suite that includes documents, spreadsheets, and slideshows in one interface. With this integration, you can use the Shimo Wiki directly within GitLab instead of the [GitLab group/project wiki](../wiki/index.md). +[Shimo](https://shimo.im/) is a productivity suite that includes documents, spreadsheets, and slideshows in one interface. +With this integration, you can use the Shimo wiki directly in GitLab instead of the [GitLab group or project wiki](../wiki/index.md). ## Configure settings in GitLab @@ -39,5 +36,3 @@ To view the Shimo Workspace from your group or project: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Shimo**. 1. On the **Shimo Workspace** page, select **Go to Shimo Workspace**. - -<!--- end_remove --> diff --git a/doc/user/project/integrations/telegram.md b/doc/user/project/integrations/telegram.md index 4e94ef76f58..94d0f887730 100644 --- a/doc/user/project/integrations/telegram.md +++ b/doc/user/project/integrations/telegram.md @@ -32,7 +32,7 @@ To configure the bot in Telegram: 1. Assign the bot `Post Messages` rights to receive events. 1. Create an identifier for the channel. - For public channels, enter a public link and copy the channel identifier (for example, `https:/t.me/MY_IDENTIFIER`). - - For private channels, use the [`getUpdates`](https://telegram-bot-sdk.readme.io/reference/getupdates) method with your API token and copy the channel identifier. + - For private channels, use the [`getUpdates`](https://telegram-bot-sdk.readme.io/reference/getupdates) method with your API token and copy the channel identifier (for example, `-2241293890657`). ## Set up the Telegram integration in GitLab diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md index 269fdf304a8..73450971434 100644 --- a/doc/user/project/integrations/webhook_events.md +++ b/doc/user/project/integrations/webhook_events.md @@ -6,8 +6,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Webhook events **(FREE ALL)** -You can configure a [webhook](webhooks.md) in your project that triggers when -an event occurs. The following events are supported. +This page lists the events that are triggered for [project webhooks](webhooks.md) and [group webhooks](webhooks.md#group-webhooks). + +For a list of events triggered for system webhooks, see [system webhooks](../../../administration/system_hooks.md). + +**Events triggered for both project and group webhooks:** Event type | Trigger ---------------------------------------------|----------------------------------------------------------------------------- @@ -20,11 +23,16 @@ Event type | Trigger [Pipeline event](#pipeline-events) | A pipeline status changes. [Job event](#job-events) | A job status changes. [Deployment event](#deployment-events) | A deployment starts, succeeds, fails, or is canceled. +[Feature flag event](#feature-flag-events) | A feature flag is turned on or off. +[Release event](#release-events) | A release is created, updated, or deleted. +[Emoji event](#emoji-events) | An emoji reaction is added or removed. + +**Events triggered for group webhooks only:** + +Event type | Trigger +---------------------------------------------|----------------------------------------------------------------------------- [Group member event](#group-member-events) | A user is added or removed from a group, or a user's access level or access expiration date changes. [Subgroup event](#subgroup-events) | A subgroup is created or removed from a group. -[Feature flag event](#feature-flag-events) | A feature flag is turned on or off. -[Release event](#release-events) | A release is created or updated. -[Emoji event](#emoji-events) | An emoji is awarded or revoked. NOTE: If an author has no public email listed in their @@ -1578,7 +1586,7 @@ Payload example: ## Group member events **(PREMIUM ALL)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/260347) in GitLab 13.7. +These events are triggered for [group webhooks](webhooks.md#group-webhooks) only. Member events are triggered when: @@ -1673,7 +1681,7 @@ Payload example: ## Subgroup events **(PREMIUM ALL)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/260419) in GitLab 13.9. +These events are triggered for [group webhooks](webhooks.md#group-webhooks) only. Subgroup events are triggered when: @@ -1790,12 +1798,15 @@ Payload example: ## Release events -Release events are triggered when a release is created or updated. +> Delete release event [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418113) in GitLab 16.5. + +Release events are triggered when a release is created, updated, or deleted. The available values for `object_attributes.action` in the payload are: - `create` - `update` +- `delete` Request header: @@ -1891,7 +1902,7 @@ On self-managed GitLab, by default this feature is available. To hide the featur NOTE: To have the `emoji_webhooks` flag enabled on GitLab.com, see [issue 417288](https://gitlab.com/gitlab-org/gitlab/-/issues/417288). -An emoji event is triggered when an emoji is awarded or revoked on: +An emoji event is triggered when an [emoji reaction](../../emoji_reactions.md) is added or removed on: - Issues - Merge requests @@ -1904,8 +1915,8 @@ An emoji event is triggered when an emoji is awarded or revoked on: The available values for `object_attributes.action` in the payload are: -- `award` -- `revoke` +- `award` to add a reaction +- `revoke` to remove a reaction Request header: diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 8e7f8bcd67d..1ecc14edcd3 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -60,7 +60,7 @@ To configure a webhook for a project or group: The URL must be percent-encoded if it contains one or more special characters. 1. In **Secret token**, enter the [secret token](#validate-payloads-by-using-a-secret-token) to validate payloads. 1. In the **Trigger** section, select the [events](webhook_events.md) to trigger the webhook. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#ssl-verification). 1. Select **Add webhook**. ## Mask sensitive portions of webhook URLs @@ -286,9 +286,9 @@ Webhook requests to your endpoint include the following headers: | Header | Description | Example | | ------ | ------ | ------ | -| `User-Agent` | In the format `"Gitlab/<VERSION>"`. | `"GitLab/15.5.0-pre"` | +| `User-Agent` | User agent in the format `"Gitlab/<VERSION>"`. | `"GitLab/15.5.0-pre"` | | `X-Gitlab-Instance` | Hostname of the GitLab instance that sent the webhook. | `"https://gitlab.com"` | -| `X-Gitlab-Webhook-UUID` | Unique ID per webhook. If a webhook request fails and retries, the second request has a new ID. | `"02affd2d-2cba-4033-917d-ec22d5dc4b38"` | +| `X-Gitlab-Webhook-UUID` | Unique ID per webhook. | `"02affd2d-2cba-4033-917d-ec22d5dc4b38"` | | `X-Gitlab-Event` | Name of the webhook type. Corresponds to [event types](webhook_events.md) but in the format `"<EVENT> Hook"`. | `"Push Hook"` | | `X-Gitlab-Event-UUID` | Unique ID per webhook that is not recursive. A hook is recursive if triggered by an earlier webhook that hit the GitLab instance. Recursive webhooks have the same value for this header. | `"13792a34-cac6-4fda-95a8-c58e00a3954e"` | diff --git a/doc/user/project/integrations/zentao.md b/doc/user/project/integrations/zentao.md index e42b0a032fd..64967e94f68 100644 --- a/doc/user/project/integrations/zentao.md +++ b/doc/user/project/integrations/zentao.md @@ -4,17 +4,14 @@ group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> - # ZenTao (deprecated) **(PREMIUM ALL)** +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338178) in GitLab 14.5. + WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/377825) in GitLab 15.7 -and is planned for removal in 16.0. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/377825) in GitLab 15.7. This change is a breaking change. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338178) in GitLab 14.5. - [ZenTao](https://www.zentao.net/) is a web-based project management platform. The following versions of ZenTao are supported: @@ -54,5 +51,3 @@ Complete these steps in GitLab: 1. Optional. Select **Test settings**. 1. Select **Save changes**. - -<!--- end_remove --> diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index 0ea49ff387f..9850dcc22fc 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -31,14 +31,10 @@ For a video overview, see [Design Management (GitLab 12.2)](https://www.youtube. Image thumbnails are stored as other uploads, and are not associated with a project but rather with a specific design model. -- Projects must use - [hashed storage](../../../administration/raketasks/storage.md#migrate-to-hashed-storage). - Newly created projects use hashed storage by default. - A GitLab administrator can verify the storage type of a project by going to **Admin Area > Projects** - and then selecting the project in question. A project can be identified as - hashed-stored if the value of the **Relative path** field contains `@hashed`. + A GitLab administrator can verify the relative path of a hashed-stored project by going to **Admin Area > Projects** + and then selecting the project in question. The **Relative path** field contains `@hashed` in its value. If the requirements are not met, you are notified in the **Designs** section. @@ -190,17 +186,11 @@ To archive multiple designs at once: ## Markdown and rich text editors for descriptions -<!-- When content_editor_on_issues flag is removed, move version notes - to "Add a design to an issue", update that topic, and delete this one. --> - > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388449) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/375172) in GitLab 16.2. +> - Feature flag `content_editor_on_issues` removed in GitLab 16.5. -FLAG: -On self-managed GitLab, by default the rich text editor is available. To hide it, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. -On GitLab.com, this feature is available. - -When this feature is enabled, you can use the Markdown and rich text editor in design descriptions. +You can use the Markdown and rich text editor in design descriptions. It's the same editor you use for comments across GitLab. ## Reorder designs diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index cb3cbf5fc36..a162c2d1709 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -214,10 +214,7 @@ To close an issue, you can either: - From any other page in the GitLab UI: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Plan > Issues**, then select your issue to view it. - 1. At the top of the issue, select **Close issue**. - -If you don't see this action at the top of an issue, your project or instance might have -enabled a feature flag to [moved it in the actions menu](#move-the-close-button-into-the-actions-menu). + 1. In the upper-right corner, select **Issue actions** (**{ellipsis_v}**) and then **Close issue**. ### Reopen a closed issue @@ -225,12 +222,9 @@ Prerequisites: - You must have at least the Reporter role for the project, be the author of the issue, or be assigned to the issue. -To reopen a closed issue, at the top of the issue, select **Reopen issue**. +To reopen a closed issue, in the upper-right corner, select **Issue actions** (**{ellipsis_v}**) and then **Reopen issue**. A reopened issue is no different from any other open issue. -If you don't see this action at the top of an issue, your project or instance might have -enabled a feature flag to [moved it in the actions menu](#move-the-close-button-into-the-actions-menu). - ### Closing issues automatically You can close issues automatically by using certain words, called a _closing pattern_, @@ -331,24 +325,6 @@ Prerequisites: Learn how to change the default [issue closing pattern](../../../administration/issue_closing_pattern.md). of your installation. -<!-- Delete when the `move_close_into_dropdown` feature flag is removed -and update steps for closing and reopening issues, incidents, and epics --> -### Move the close button into the actions menu - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125173) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `move_close_into_dropdown`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `move_close_into_dropdown`. -On GitLab.com, this feature is not available. - -When this feature flag is enabled, in the upper-right corner, -**Issue actions** (**{ellipsis_v}**) contains the **Close issue** and **Reopen issue** actions. - -In GitLab 16.2 and later, similar action menus are also available on incidents and epics. - -When this feature flag is disabled, **Close issue** and **Reopen issue** are -on the top bar, outside of the actions menu. - ## Change the issue type Prerequisites: diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index c365bfa5a52..f2ecfc1f24b 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -73,7 +73,7 @@ then issues with a milestone without a due date. ## Sorting by popularity When you sort by **Popularity**, the issue order changes to sort descending by the -number of upvotes ([emoji reactions](../../award_emojis.md) with the "thumbs up") +number of upvotes ([emoji reactions](../../emoji_reactions.md) with the "thumbs up") on each issue. You can use this to identify issues that are in high demand. The total number of votes is not summed up. An issue with 18 upvotes and 5 diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 86d21d07950..eb872a24767 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -28,7 +28,6 @@ You can use two types of labels in GitLab: ## Assign and unassign labels -> - Unassigning labels with the **X** button [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216881) in GitLab 13.5. > - Real-time updates in the sidebar [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241538) in GitLab 14.10 with a [feature flag](../../administration/feature_flags.md) named `realtime_labels`, disabled by default. > - Real-time updates in the sidebar [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357370#note_991987201) in GitLab 15.1. > - Real-time updates in the sidebar [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/357370) in GitLab 15.5. @@ -65,8 +64,6 @@ You can also assign and unassign labels with [quick actions](quick_actions.md): ### View project labels -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241990) in GitLab 13.5: the label list in a project also shows all inherited labels. - To view the **project's labels**: 1. On the left sidebar, select **Search or go to** and find your project. @@ -184,7 +181,8 @@ To edit a **project** label: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Manage > Labels**. -1. Next to the label you want to edit, select **Edit** (**{pencil}**). +1. Next to the label you want to edit, select the vertical ellipsis (**{ellipsis_v}**), and then select **Edit**. +1. Select **Save changes**. ### Edit a group label @@ -192,7 +190,8 @@ To edit a **group** label: 1. On the left sidebar, select **Search or go to** and find your group. 1. Select **Manage > Labels**. -1. Next to the label you want to edit, select **Edit** (**{pencil}**). +1. Next to the label you want to edit, select the vertical ellipsis (**{ellipsis_v}**), and then select **Edit**. +1. Select **Save changes**. ## Delete a label @@ -210,12 +209,7 @@ To delete a **project** label: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Manage > Labels**. -1. Either: - - - Next to the **Subscribe** button, select (**{ellipsis_v}**). - - Next to the label you want to edit, select **Edit** (**{pencil}**). - -1. Select **Delete**. +1. Next to the **Subscribe** button, select (**{ellipsis_v}**), and then select **Delete**. ### Delete a group label @@ -232,8 +226,6 @@ To delete a **group** label: ## Promote a project label to a group label -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231472) in GitLab 13.6: promoting a project label keeps that label's ID and changes it into a group label. Previously, promoting a project label created a new group label with a new ID and deleted the old label. - You might want to make a project label available for other projects in the same group. Then, you can promote the label to a group label. @@ -455,6 +447,34 @@ The labels higher in the list get higher priority. To learn what happens when you sort by priority or label priority, see [Sorting and ordering issue lists](issues/sorting_issue_lists.md). +## Lock labels when a merge request is merged **(FREE SAAS BETA)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408676) in GitLab 16.3 [with a flag](../../administration/feature_flags.md) named `enforce_locked_labels_on_merge`. This feature is [Beta](../../policy/experiment-beta-support.md). + +FLAG: +On self-managed GitLab, this feature is not available. +On GitLab.com, this feature is only available for use by GitLab Inc. To make it available per group or per project, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `enforce_locked_labels_on_merge`. + +To comply with certain auditing requirements, you can set a label to be locked. +When a merge request with locked labels gets merged, nobody can remove them from the MR. + +When you add locked labels to issues or epics, they behave like regular labels. + +Prerequisites: + +- You must have at least the Reporter role for the project or group. + +WARNING: +After you set a label as locked, nobody can undo it or delete the label. + +To set a label to get locked on merge: + +1. On the left sidebar, select **Search or go to** and find your group or project. +1. Select **Manage > Labels**. +1. Next to the label you want to edit, select the vertical ellipsis (**{ellipsis_v}**), and then select **Edit**. +1. Select the **Lock label after a merge request is merged** checkbox. +1. Select **Save changes**. + ## Related topics Practice working with labels in the following tutorials: diff --git a/doc/user/project/merge_requests/ai_in_merge_requests.md b/doc/user/project/merge_requests/ai_in_merge_requests.md index 304717cf9fc..c29060bf44b 100644 --- a/doc/user/project/merge_requests/ai_in_merge_requests.md +++ b/doc/user/project/merge_requests/ai_in_merge_requests.md @@ -4,13 +4,13 @@ group: Code Review info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# AI/ML powered features in merge requests +# GitLab Duo in merge requests **(ULTIMATE SAAS EXPERIMENT)** AI-assisted features in merge requests are designed to provide contextually relevant information during the lifecycle of a merge request. -Additional information on enabling these features and maturity can be found in our [AI/ML Overview](../../ai_features.md). +Additional information on enabling these features and maturity can be found in our [GitLab Duo overview](../../ai_features.md). -## Fill in merge request templates **(ULTIMATE SAAS)** +## Fill in merge request templates > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10591) in GitLab 16.3 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). @@ -18,7 +18,7 @@ This feature is an [Experiment](../../../policy/experiment-beta-support.md) on G Merge requests in projects often have [templates](../description_templates.md#create-a-merge-request-template) defined that need to be filled out. This helps reviewers and other users understand the purpose and changes a merge request might propose. -When creating a merge request you can now choose to generate a description for the merge request based on the contents of the template. This fills in the template and replaces the current contents of the description. +When creating a merge request, GitLab Duo can generate a description for the merge request based on the contents of the template. This fills in the template and replaces the current contents of the description. To generate the description: @@ -36,27 +36,29 @@ Provide feedback on this experimental feature in [issue 416537](https://gitlab.c - Contents of the description - Diff of changes between the source branch's head and the target branch -## Summarize merge request changes **(ULTIMATE SAAS)** +## Summarize merge request changes > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10401) in GitLab 16.2 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. -These summaries are automatically generated. They are available on the merge request page in the **Merge request summaries** dialog, the To-Do list, and in email notifications. +GitLab Duo Merge request summaries are available on the merge request page in: + +- The **Merge request summaries** dialog. +- The To-Do list. +- Email notifications. Provide feedback on this experimental feature in [issue 408726](https://gitlab.com/gitlab-org/gitlab/-/issues/408726). **Data usage**: The diff of changes between the source branch's head and the target branch is sent to the large language model. -## Summarize my merge request review **(ULTIMATE SAAS)** +## Summarize my merge request review > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10466) in GitLab 16.0 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. -When you've completed your review of a merge request and are ready to [submit your review](reviews/index.md#submit-a-review), you can have a summary generated for you. - -To generate the summary: +When you've completed your review of a merge request and are ready to [submit your review](reviews/index.md#submit-a-review), generate a GitLab Duo Code review summary: 1. When you are ready to submit your review, select **Finish review**. 1. Select **AI Actions** (**{tanuki}**). @@ -72,15 +74,15 @@ Provide feedback on this experimental feature in [issue 408991](https://gitlab.c - Draft comment's text -## Suggested merge or squash commit message **(ULTIMATE SAAS)** +## Generate messages for merge or squash commits > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10453) in GitLab 16.2 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. -When preparing to merge your merge request you may wish to edit the squash or merge commit message that will be used. +When preparing to merge your merge request you may wish to edit the proposed squash or merge commit message. -To generate a commit message: +To generate a commit message with GitLab Duo: 1. Select the **Edit commit message** checkbox on the merge widget. 1. Select **Create AI-generated commit message**. @@ -93,13 +95,13 @@ Provide feedback on this experimental feature in [issue 408994](https://gitlab.c - Contents of the file - The filename -## Generate suggested tests in merge requests **(ULTIMATE SAAS)** +## Generate suggested tests in merge requests > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10366) in GitLab 16.0 as an [Experiment](../../../policy/experiment-beta-support.md#experiment). -This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `text-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. +This feature is an [Experiment](../../../policy/experiment-beta-support.md) on GitLab.com that is using Google's Vertex service and the `code-bison` model. It requires the [group-level third-party AI features setting](../../group/manage.md#enable-third-party-ai-features) to be enabled. -In a merge request, you can get a list of suggested tests for the file you are reviewing. This functionality can help determine if appropriate test coverage has been provided, or if you need more coverage for your project. +Use GitLab Duo Test generation in a merge request to see a list of suggested tests for the file you are reviewing. This functionality can help determine if appropriate test coverage has been provided, or if you need more coverage for your project. View a [click-through demo](https://go.gitlab.com/Xfp0l4). diff --git a/doc/user/project/merge_requests/dependencies.md b/doc/user/project/merge_requests/dependencies.md index e208785fd63..89305e65dfb 100644 --- a/doc/user/project/merge_requests/dependencies.md +++ b/doc/user/project/merge_requests/dependencies.md @@ -145,10 +145,9 @@ information, read [issue #12549](https://gitlab.com/gitlab-org/gitlab/-/issues/1 ### Complex merge order dependencies are unsupported -If you attempt to create an indirect, nested dependency, GitLab shows one of these error messages: +If you attempt to create an indirect, nested dependency, GitLab shows the error message: -- Dependencies failed to save: Blocked merge request cannot block others -- Dependencies failed to save: Blocking merge request cannot itself be blocked +- Dependencies failed to save: Dependency chains are not supported GitLab supports direct dependencies between merge requests, but does not support [indirect (nested) dependencies](https://gitlab.com/gitlab-org/gitlab/-/issues/11393). diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index ed762979ff1..22cd8f9b89e 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -302,12 +302,9 @@ For a web developer writing a webpage for your company's website: ## Filter activity in a merge request > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115383) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `mr_activity_filters`. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/387070) in GitLab 16.0. Available to GitLab team members only. - -FLAG: -On self-managed GitLab, by default this feature is not available. -To make it available per user, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `mr_activity_filters` for individual or groups of users. -On GitLab.com, this feature is enabled for GitLab team members only. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/387070) in GitLab 16.0. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/126998) in GitLab 16.3 by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/132355) in GitLab 16.5. Feature flag `mr_activity_filters` removed. To understand the history of a merge request, filter its activity feed to show you only the items that are relevant to you. @@ -394,6 +391,20 @@ with a new push. Threads are now resolved if a push makes a diff section outdated. Threads on lines that don't change and top-level resolvable threads are not resolved. +## Move notifications and to-dos **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/132678) in GitLab 16.5 [with a flag](../../../administration/feature_flags.md) named `notifications_todos_buttons`. Disabled by default. +> - [Issues, incidents](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133474), and [epics](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/133881) also updated. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `notifications_todos_buttons`. +On GitLab.com, this feature is not available. + +When this feature flag is enabled, the notifications and to-do item buttons are moved to the upper right corner of the page. + +- On merge requests, these buttons are located to the far right of the tabs. +- On issues, incidents, and epics, these buttons are located at the top of the right sidebar. + ## Related topics - [Create a merge request](creating_merge_requests.md) diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 77dcb269071..699c79806f0 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -94,9 +94,14 @@ To enable this setting: 1. Select **Settings > Merge requests**. 1. Scroll to **Merge checks**, and select **Pipelines must succeed**. This setting also prevents merge requests from being merged if there is no pipeline, - which can [conflict with some rules](#merge-requests-dont-merge-when-successful-pipeline-is-required). + which can [conflict with some rules](#merge-request-cannot-be-merged-despite-no-failed-pipeline). 1. Select **Save**. +If [multiple pipeline types run for the same merge request](#merge-request-can-still-be-merged-despite-a-failed-pipeline), +merge request pipelines take precedence over other pipeline types. For example, +an older but successful merge request pipeline allows a merge request to be merged, +despite a newer but failed branch pipeline. + ### Allow merge after skipped pipelines > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211482) in GitLab 13.1. @@ -120,44 +125,30 @@ To change this behavior: ## Troubleshooting -### Merge requests don't merge when successful pipeline is required - -If you require a successful pipeline for a merge, this setting can conflict with some -use cases that do not generate pipelines, such as [`only/except`](../../../ci/yaml/index.md#only--except) -or [`rules`](../../../ci/yaml/index.md#rules). Ensure your project -[runs a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) for -every merge request, and that the pipeline is successful. - -### Ensure test parity between pipeline types - -If a merge request triggers both a branch pipeline and a merge request pipeline, -the success or failure of only the *merge request pipeline* is checked. -If the merge request pipeline contains fewer jobs than the branch pipeline, -it could allow code that fails tests to be merged, like in this example: - -```yaml -branch-pipeline-job: - rules: - - if: $CI_PIPELINE_SOURCE == "push" - script: - - echo "Testing happens here." - -merge-request-pipeline-job: - rules: - - if: $CI_PIPELINE_SOURCE == "merge_request_event" - script: - - echo "No testing happens here. This pipeline always succeeds, and enables merge." - - echo true -``` - -Instead, use branch (`push`) pipelines or merge request pipelines, when possible. -For details on avoiding two pipelines for a single merge request, read the -[`rules` documentation](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines). - -### Merged results pipeline allows merge, despite a failed branch pipeline - -When [the **Pipelines must succeed** setting](#require-a-successful-pipeline-for-merge) -is combined with -[the **Merged results pipelines** feature](../../../ci/pipelines/merged_results_pipelines.md), -failed branch pipeline may be ignored. -[Issue 385841](https://gitlab.com/gitlab-org/gitlab/-/issues/385841) is open to track this. +### Merge request cannot be merged despite no failed pipeline + +In some cases, you can [require a successful pipeline for merge](#require-a-successful-pipeline-for-merge), +but be unable to merge a merge request with no failed pipelines. The setting requires +the existence of a successful pipeline, not the absence of failed pipelines. If the merge request +has no pipelines at all, it is not considered to have a successful pipeline and cannot be merged. + +When the setting is enabled, use [`rules`](../../../ci/yaml/index.md#rules) or [`workflow:rules`](../../../ci/yaml/index.md#workflowrules) +to ensure pipelines run for every merge request. + +### Merge request can still be merged despite a failed pipeline + +In some cases, you can [require a successful pipeline for merge](#require-a-successful-pipeline-for-merge), +but still merge a merge request with a failed pipeline. + +Merge request pipelines have the highest priority for the **Pipelines must succeed** setting. +If multiple pipeline types run for the same merge request, only the merge request pipelines +are checked for success. + +Multiple pipeline types in the same merge request can be caused by: + +- A [`rules`](../../../ci/yaml/index.md#rules) configuration that causes [duplicate pipelines](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines): + one merge request pipeline and one branch pipeline. In this case, the status of the + latest merge request pipeline determines if a merge request can be merged, not the branch pipeline. +- Pipelines triggered by external tools that target the same branch as the merge request. + +In all cases, update your CI/CD configuration to prevent multiple pipeline types for the same merge request. diff --git a/doc/user/project/merge_requests/reviews/data_usage.md b/doc/user/project/merge_requests/reviews/data_usage.md index 24e3b6a5667..b4b9b19c932 100644 --- a/doc/user/project/merge_requests/reviews/data_usage.md +++ b/doc/user/project/merge_requests/reviews/data_usage.md @@ -9,7 +9,7 @@ type: index, reference ## How it works -Suggested Reviewers is the first user-facing GitLab machine learning (ML) powered feature. It leverages a project's contribution graph to generate suggestions. This data already exists within GitLab including merge request metadata, source code files, and GitLab user account metadata. +GitLab Duo Suggested Reviewers is the first user-facing GitLab machine learning (ML) powered feature. It leverages a project's contribution graph to generate suggestions. This data already exists within GitLab including merge request metadata, source code files, and GitLab user account metadata. ### Enabling the feature diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index c09071e856c..0a3efa38440 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -40,12 +40,12 @@ GitLab Duo Suggested Reviewers also integrates with Code Owners, profile status, For more information, see [Data usage in GitLab Duo Suggested Reviewers](data_usage.md). -### Enable suggested reviewers +### Enable Suggested Reviewers -Project Maintainers or Owners can enable suggested reviewers by visiting +Project Maintainers or Owners can enable Suggested Reviewers by visiting the [project settings](../../settings/index.md). -Enabling suggested reviewers triggers GitLab to create an ML model for your +Enabling Suggested Reviewers triggers GitLab to create an ML model for your project that is used to generate reviewers. The larger your project, the longer this process can take. Usually, the model is ready to generate suggestions within a few hours. @@ -199,7 +199,7 @@ If you have a review in progress, you can also add a comment from the **Overview When editing the **Reviewers** field in a new or existing merge request, GitLab displays the name of the matching [approval rule](../approvals/rules.md) -below the name of each suggested reviewer. [Code Owners](../../codeowners/index.md) are displayed as `Codeowner` without group detail. +below the name of each reviewer. [Code Owners](../../codeowners/index.md) are displayed as `Codeowner` without group detail. This example shows reviewers and approval rules when creating a new merge request: diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md index 2b046399c4e..90a276dc303 100644 --- a/doc/user/project/merge_requests/reviews/suggestions.md +++ b/doc/user/project/merge_requests/reviews/suggestions.md @@ -71,10 +71,7 @@ suggestion. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388449) in GitLab 16.1 [with a flag](../../../../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/375172) in GitLab 16.2. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../../administration/feature_flags.md) named `content_editor_on_issues`. -On GitLab.com, this feature is available. +> - Feature flag `content_editor_on_issues` removed in GitLab 16.5. When you insert suggestions, you can use the WYSIWYG [rich text editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/) to move diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index f87d379b974..698078351e2 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -94,7 +94,7 @@ Filling in the form and selecting the **Add status check** button creates a new ### Update a status check service -Within the **Status checks** sub-section, select the **Edit** button +Within the **Status checks** sub-section, select **Edit** (**{pencil}**) next to the status check you want to edit. The **Update status check** form is then shown. @@ -137,7 +137,7 @@ you can select the **All branches** option. ## Delete a status check service -Within the **Status checks** sub-section, select the **Remove...** button +Within the **Status checks** sub-section, select **Remove** (**{remove}**) next to the status check you want to delete. The **Remove status check?** modal is then shown. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index d8e4fce41ef..1c2b342f5d3 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -19,6 +19,9 @@ To use one or more custom domain names: - Add a [custom **root domain** or a **subdomain**](#set-up-a-custom-domain). - Add [SSL/TLS certification](#adding-an-ssltls-certificate-to-pages). +WARNING: +You cannot verify the [most popular public email domains](../../../../user/group/access_and_permissions.md#restrict-group-access-by-domain). + ## Set up a custom domain To set up Pages with a custom domain name, read the requirements and steps below. diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 097c726d163..16967a3a46e 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -146,9 +146,10 @@ To auto-format this table, use the VS Code Markdown Table formatter: `https://do |:--------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------| | `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Assign one or more users. | | `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Assign yourself. | +| `/add_child <work_item>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Add child to `<work_item>`. The `<work_item>` value should be in the format of `#iid`, `group/project#iid`, or a URL to a work item. Multiple work items can be added as children at the same time. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420797) in GitLab 16.5. | | `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle an emoji reaction. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412275) in GitLab 16.5 | | `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line creates a specific type of to-do item notification. | -| `/checkin_reminder` | **{dotted-circle}** No| **{check-circle}** Yes | **{dotted-circle}** No | Set checkin reminder cadence. Options are `weekly`, `twice-monthly`, `monthly`, `never`. This action is behind a feature flag. | +| `/checkin_reminder <cadence>` | **{dotted-circle}** No| **{check-circle}** Yes | **{dotted-circle}** No | Schedule [check-in reminders](../okrs.md#schedule-okr-check-in-reminders). Options are `weekly`, `twice-monthly`, `monthly`, or `never` (default). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422761) in GitLab 16.4 with flags named `okrs_mvc` and `okr_checkin_reminders`. | | `/clear_health_status` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). | | `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | | `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | @@ -162,6 +163,7 @@ To auto-format this table, use the VS Code Markdown Table formatter: `https://do | `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | | `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Remove due date. | | `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | +| `/set_parent <work_item>` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Set parent work item to `<work_item>`. The `<work_item>` value should be in the format of `#iid`, `group/project#iid`, or a URL to a work item. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420798) in GitLab 16.5. | | `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. | | `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/420796) in GitLab 16.4 | | `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index f33d443cd7f..30ddf8d3230 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -286,6 +286,47 @@ To do this: 1. Select **Delete merged branches**. 1. In the dialog, enter the word `delete` to confirm, then select **Delete merged branches**. +## Configure rules for target branches **(PREMIUM ALL)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/127115) in GitLab 16.4 [with a flag](../../../../administration/feature_flags.md) named `target_branch_rules_flag`. Enabled by default. + +Some projects use multiple long-term branches for development, like `develop` and `qa`. +In these projects, you might want to keep `main` as the default branch, but expect +merge requests to target `develop` or `qa` instead. Target branch rules help ensure +merge requests target the appropriate development branch for your project. + +When you create a merge request, the rule checks the name of the branch. If the +branch name matches the rule, the merge request targets the branch you specify +in the rule. If the branch name does not match, the merge request targets the +default branch of the project. + +Prerequisites: + +- You must have at least the Maintainer role. + +To create a target branch rule: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > Merge requests**. +1. Select **Add target branch rule**. +1. For **Rule name**, provide a string or wild card to compare against branch names. +1. Select the **Target branch** to use when the branch name matches the **Rule name**. +1. Select **Save**. + +## Delete a target branch rule + +When you remove a target branch rule, existing merge requests remain unchanged. + +Prerequisites: + +- You must have at least the Maintainer role. + +To do this: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > Merge requests**. +1. Select **Delete** on the rule you want to delete. + ## Related topics - [Protected branches](../../protected_branches.md) diff --git a/doc/user/project/repository/code_suggestions/index.md b/doc/user/project/repository/code_suggestions/index.md index 4f0c0b2c9a6..151792089ce 100644 --- a/doc/user/project/repository/code_suggestions/index.md +++ b/doc/user/project/repository/code_suggestions/index.md @@ -9,6 +9,7 @@ type: index, reference > - [Introduced support for Google Vertex AI Codey APIs](https://gitlab.com/groups/gitlab-org/-/epics/10562) in GitLab 16.1. > - [Removed support for GitLab native model](https://gitlab.com/groups/gitlab-org/-/epics/10752) in GitLab 16.2. +> - [Introduced support for Code Generation](https://gitlab.com/gitlab-org/gitlab/-/issues/415583) in GitLab 16.3. WARNING: This feature is in [Beta](../../../../policy/experiment-beta-support.md#beta). @@ -23,18 +24,45 @@ GitLab Duo Code Suggestions are available: - In the GitLab WebIDE. <div class="video-fallback"> - <a href="https://www.youtube.com/watch?v=WnxBYxN2-p4">View an end-to-end demo of Code Suggestions in VS Code</a>. + <a href="https://youtu.be/wAYiy05fjF0">View how to setup and use GitLab Duo Code Suggestions</a>. </div> <figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe> + <iframe src="https://www.youtube-nocookie.com/embed/wAYiy05fjF0" frameborder="0" allowfullscreen> </iframe> </figure> -Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). +During Beta, usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). +## Use Code Suggestions + +Prerequisites: + +- Code Suggestions must be enabled for [SaaS](saas.md#enable-code-suggestions) or for [self-managed](self_managed.md#enable-code-suggestions-on-self-managed-gitlab). +- You must have installed and configured a [supported IDE editor extension](index.md#supported-editor-extensions). + +To use Code Suggestions: + +1. Author your code. As you type, suggestions are displayed. Code Suggestions, depending on the cursor position, either provides code snippets or completes the current line. +1. Describe the requirements in natural language. Be concise and specific. Code Suggestions generates functions and code snippets as appropriate. +1. To accept a suggestion, press <kbd>Tab</kbd>. +1. To ignore a suggestion, keep typing as you usually would. +1. To explicitly reject a suggestion, press <kbd>esc</kbd>. + +Things to remember: + +- AI is non-deterministic, so you may not get the same suggestion every time with the same input. +- Just like product requirements, writing clear, descriptive, and specific tasks results in quality generated code. + +### Progressive enhancement + +This feature is designed as a progressive enhancement to developer's IDEs. +Code Suggestions offer a completion if a suitable recommendation is provided to the user in a timely matter. +In the event of a connection issue or model inference failure, the feature gracefully degrades. +Code Suggestions do not prevent you from writing code in your IDE. + ## Supported languages -The best results from Code Suggestions are expected [for languages the Google Vertex AI Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_coding_languages) directly support: +The best results from Code Suggestions are expected for languages that [Anthropic Claude](https://www.anthropic.com/product) and the [Google Vertex AI Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_coding_languages) directly support: - C++ - C# @@ -51,39 +79,33 @@ The best results from Code Suggestions are expected [for languages the Google Ve - Swift - TypeScript -### Supported code infrastructure interfaces - -Code Suggestions includes [Google Vertex AI Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_code_infrastructure_interfaces) support for the following infrastructure as code interfaces: - -- Google Cloud CLI -- Kubernetes Resource Model (KRM) -- Terraform - -Suggestion quality for other languages and using natural language code comments to request completions may not yet result in high-quality suggestions. - ### Supported languages in IDEs Editor support for languages is documented in the following table. | Language | VS Code | JetBrains IDEs | Visual Studio | Neovim | |------------------|------------------------|------------------------|------------------------|--------| -| C++ | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | **{check-circle}** Yes | +| C++ | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | C# | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | -| Go | **{check-circle}** Yes | **{check-circle}** Yes (IDEA Ultimate / GoLand) | **{check-circle}** Yes | **{check-circle}** Yes | -| Google SQL | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | **{check-circle}** Yes | +| Go | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | +| Google SQL | **{dotted-circle}** No | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Java | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | JavaScript | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Kotlin | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | -| PHP | **{check-circle}** Yes | **{check-circle}** Yes (IDEA Ultimate) | **{check-circle}** Yes | **{check-circle}** Yes | +| PHP | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Python | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | -| Ruby | **{check-circle}** Yes | **{check-circle}** Yes (IDEA Ultimate / RubyMine) | **{check-circle}** Yes | **{check-circle}** Yes | +| Ruby | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Rust | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Scala | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Swift | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | TypeScript | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | | Google Cloud | **{dotted-circle}** No | **{dotted-circle}** No | **{dotted-circle}** No | **{dotted-circle}** No | | Kubernetes Resource Model (KRM) | **{dotted-circle}** No | **{dotted-circle}** No | **{dotted-circle}** No | **{dotted-circle}** No | -| Terraform | **{check-circle}** Yes (Requires third-party extension providing Terraform support) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes (Requires third-party extension providing the `terraform` file type) | +| Terraform | **{check-circle}** Yes (Requires third-party extension providing Terraform support) | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes (Requires third-party extension providing the `terraform` file type) | + +NOTE: +Some languages are not supported in all JetBrains IDEs, or may require additional +plugin support. Refer to the JetBrains documentation for specifics on your IDE. ## Supported editor extensions @@ -104,13 +126,12 @@ This improvement should result in: ## Code Suggestions data usage -Code Suggestions is a generative artificial intelligence (AI) model. +Code Suggestions is powered by a generative AI model. -Your personal access token enables a secure API connection to GitLab.com. -This API connection securely transmits a context window from your IDE/editor to the Code Suggestions GitLab hosted service which calls Google Vertex AI Codey APIs, -and the generated suggestion is transmitted back to your IDE/editor. +Your personal access token enables a secure API connection to GitLab.com or to your GitLab instance. +This API connection securely transmits a context window from your IDE/editor to the [GitLab AI Gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist), a GitLab hosted service. The gateway calls the large language model APIs, and then the generated suggestion is transmitted back to your IDE/editor. -GitLab currently leverages [Google Cloud's Vertex AI Codey API models](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview). Learn more about Google Vertex AI Codey APIs [Data Governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance). +GitLab selects the best-in-class large-language models for specific tasks. We use [Google Vertex AI Code Models](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) and [Anthropic Claude](https://www.anthropic.com/product) for Code Suggestions. ### Telemetry @@ -127,48 +148,16 @@ For self-managed instances that have enabled Code Suggestions and SaaS accounts, ### Inference window context -Code Suggestions currently inferences against the currently opened file and has a context window of 2,048 tokens and 8,192 character limits. This limit includes content before and after the cursor, the file name, and the extension type. -Learn more about Google Vertex AI [code-gecko](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models). - -The maximum number of tokens that is generated in the response is default 64. A token is approximately four characters. 100 tokens correspond to roughly 60-80 words. -Learn more about Google Vertex AI [`code-gecko`](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/code-completion). +Code Suggestions inferences against the currently opened file, the content before and after the cursor, the filename, and the extension type. For more information on possible future context expansion to improve the quality of suggestions, see [epic 11669](https://gitlab.com/groups/gitlab-org/-/epics/11669). ### Training data -Code Suggestions are routed through Google Vertex AI Codey APIs. Learn more about Google Vertex AI Codey APIs [Data Governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance) and [Responsible AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai). - -Google Vertex AI Codey APIs are not trained on private non-public GitLab customer or user data. - -Google has [shared the following](https://ai.google/discover/foundation-models/) about the data Codey models are trained on: - -> Codey is our family of foundational coding models built on PaLM 2. Codey was fine-tuned on a large dataset of high quality, permissively licensed code from external sources - -## Progressive enhancement - -This feature is designed as a progressive enhancement to developer's IDEs. -Code Suggestions offer a completion if the machine learning engine can generate a recommendation. -In the event of a connection issue or model inference failure, the feature gracefully degrades. -Code Suggestions do not prevent you from writing code in your IDE. +GitLab does not train generative AI models based on private (non-public) data. The vendors we work with also do not train models based on private data. -### Internet connectivity +For more information on GitLab Code Suggestions data [sub-processors](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors), see: -Code Suggestions does not work with offline environments. - -To use Code Suggestions: - -- On GitLab.com, you must have an internet connection and be able to access GitLab. -- In GitLab 16.1 and later, on self-managed GitLab, you must have an internet connection. - -### Model accuracy and quality - -Code Suggestions can generate low-quality, incomplete, and possibly insecure code. -We strongly encourage all beta users to leverage GitLab native -[Code Quality Scanning](../../../../ci/testing/code_quality.md) and -[Security Scanning](../../../application_security/index.md) capabilities. - -GitLab currently does not retrain Google Vertex AI Codey APIs. GitLab makes no claims -to the accuracy or quality of Code Suggestions generated by Google Vertex AI Codey API. -Read more about [Google Vertex AI foundation model capabilities](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models). +- Google Vertex AI Codey APIs [data governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance) and [responsible AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai). +- Anthropic Claude's [constitution](https://www.anthropic.com/index/claudes-constitution). ## Known limitations @@ -181,12 +170,6 @@ However, Code Suggestions may generate suggestions that are: - Insecure code - Offensive or insensitive -We are also aware of specific situations that can produce unexpected or incoherent results including: - -- Suggestions written in the middle of existing functions, or "fill in the middle." -- Suggestions based on natural language code comments. -- Suggestions that mixed programming languages in unexpected ways. - ## Feedback Report issues in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/405152). diff --git a/doc/user/project/repository/code_suggestions/saas.md b/doc/user/project/repository/code_suggestions/saas.md index 174c227b6fe..ac64aba4335 100644 --- a/doc/user/project/repository/code_suggestions/saas.md +++ b/doc/user/project/repository/code_suggestions/saas.md @@ -22,12 +22,14 @@ Learn about [data usage when using Code Suggestions](index.md#code-suggestions-d > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121079) in GitLab 16.1 as [Beta](../../../../policy/experiment-beta-support.md#beta). -You can enable Code Suggestions for an individual or group: +You must enable Code Suggestions for both your user account and your group: - [Enable Code Suggestions for all group members](../../../group/manage.md#enable-code-suggestions). (You must be a group owner). - [Enable Code Suggestions for your own account](../../../profile/preferences.md#enable-code-suggestions). -The group setting takes precedence over the user setting. +NOTE: +If you are having issues enabling Code Suggestions, view the +[troubleshooting guide](troubleshooting.md#code-suggestions-arent-displayed). ## Use Code Suggestions @@ -36,19 +38,4 @@ Prerequisites: - Ensure Code Suggestions is enabled for your user and group. - You must have installed and configured a [supported IDE editor extension](index.md#supported-editor-extensions). -To use Code Suggestions: - -1. Author your code. As you type, suggestions are displayed. Depending on the cursor position, the extension either: - - - Provides entire code snippets, like generating functions. - - Completes the current line. - -1. To accept a suggestion, press <kbd>Tab</kbd>. - -Suggestions are best when writing new code. Editing existing functions or 'fill in the middle' of a function may not perform as expected. - -GitLab is making improvements to the Code Suggestions to improve the quality. AI is non-deterministic, so you may not get the same suggestion every time with the same input. - -This feature is currently in [Beta](../../../../policy/experiment-beta-support.md#beta). -Code Suggestions depends on both Google Vertex AI Codey APIs and the GitLab Code Suggestions service. We have built this feature to gracefully degrade and have controls in place to allow us to -mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion. +[Use Code Suggestions](index.md#use-code-suggestions). diff --git a/doc/user/project/repository/code_suggestions/self_managed.md b/doc/user/project/repository/code_suggestions/self_managed.md index 3c149604086..ee501212027 100644 --- a/doc/user/project/repository/code_suggestions/self_managed.md +++ b/doc/user/project/repository/code_suggestions/self_managed.md @@ -156,22 +156,7 @@ Prerequisites: - Code Suggestions must be enabled [for the instance](#enable-code-suggestions-on-self-managed-gitlab). - You must have installed and configured a [supported IDE editor extension](index.md#supported-editor-extensions). -To use Code Suggestions: - -1. Author your code. As you type, suggestions are displayed. Depending on the cursor position, the extension either: - - - Provides entire code snippets, like generating functions. - - Completes the current line. - -1. To accept a suggestion, press <kbd>Tab</kbd>. - -Suggestions are best when writing new code. Editing existing functions or 'fill in the middle' of a function may not perform as expected. - -GitLab is making improvements to the Code Suggestions to improve the quality. AI is non-deterministic, so you may not get the same suggestion every time with the same input. - -This feature is currently in [Beta](../../../../policy/experiment-beta-support.md#beta). -Code Suggestions depends on both Google Vertex AI Codey APIs and the GitLab Code Suggestions service. We have built this feature to gracefully degrade and have controls in place to allow us to -mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion. +[Use Code Suggestions](index.md#use-code-suggestions). ### Data privacy diff --git a/doc/user/project/repository/code_suggestions/troubleshooting.md b/doc/user/project/repository/code_suggestions/troubleshooting.md index c0cdb3cc32d..2faf20b3035 100644 --- a/doc/user/project/repository/code_suggestions/troubleshooting.md +++ b/doc/user/project/repository/code_suggestions/troubleshooting.md @@ -18,7 +18,8 @@ In GitLab, ensure Code Suggestions is enabled: - [For your user account](../../../profile/preferences.md#enable-code-suggestions). - [For *all* top-level groups your account belongs to](../../../group/manage.md#enable-code-suggestions). If you don't have a role that lets you view the top-level group's settings, contact a group owner. -To confirm that your account is enabled, go to [https://gitlab.com/api/v4/ml/ai-assist](https://gitlab.com/api/v4/ml/ai-assist). A response of `user_is_allowed` should return `true`. +To confirm that your account is enabled, go to [https://gitlab.com/api/v4/ml/ai-assist](https://gitlab.com/api/v4/ml/ai-assist). The `user_is_allowed` key should have should have a value of `true`. +A `404 Not Found` result is returned if either of the previous conditions is not met. ### Code Suggestions not displayed in VS Code or GitLab WebIDE diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index a304a8d108d..ddc650c3924 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -15,6 +15,11 @@ A fork is a personal copy of the repository and all its branches, which you crea in a namespace of your choice. Make changes in your own fork and submit them through a merge request to the repository you don't have access to. +The forked project uses a +[deduplication strategy](../../../development/git_object_deduplication.md) +to have a potentially smaller storage space than the source project. Forked projects +can access the object pool connected to the source project. + ## Create a fork > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) a new form in GitLab 13.11 [with a flag](../../../user/feature_flags.md) named `fork_project_form`. Disabled by default. diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index e97892458b4..96b92a057cf 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -100,7 +100,7 @@ from the GitLab user interface. Prerequisites: -- The [Jetbrains Toolbox App](https://www.jetbrains.com/toolbox-app/) must be also be installed. +- The [JetBrains Toolbox App](https://www.jetbrains.com/toolbox-app/) must be also be installed. To do this: diff --git a/doc/user/project/repository/managing_large_repositories.md b/doc/user/project/repository/managing_large_repositories.md index 1d5127b5e08..1fedd8da20c 100644 --- a/doc/user/project/repository/managing_large_repositories.md +++ b/doc/user/project/repository/managing_large_repositories.md @@ -1,411 +1,11 @@ --- -stage: Systems -group: Gitaly -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'monorepos/index.md' +remove_date: '2023-12-17' --- -# Managing monorepos +This document was moved to [another location](monorepos/index.md). -Monorepos have become a regular part of development team workflows. While they have many advantages, monorepos can present performance challenges -when using them in GitLab. Therefore, you should know: - -- What repository characteristics can impact performance. -- Some tools and steps to optimize monorepos. - -## Impact on performance - -Because GitLab is a Git-based system, it is subject to similar performance -constraints as Git when it comes to large repositories that are gigabytes in -size. - -Monorepos can be large for [many reasons](https://about.gitlab.com/blog/2022/09/06/speed-up-your-monorepo-workflow-in-git/#characteristics-of-monorepos). - -Large repositories pose a performance risk performance when used in GitLab, especially if a large monorepo receives many clones or pushes a day, which is common for them. - -Git itself has performance limitations when it comes to handling -monorepos. - -[Gitaly](https://gitlab.com/gitlab-org/gitaly) is our Git storage service built -on top of [Git](https://git-scm.com/). This means that any limitations of -Git are experienced in Gitaly, and in turn by end users of GitLab. - -## Profiling repositories - -Large repositories generally experience performance issues in Git. Knowing why -your repository is large can help you develop mitigation strategies to avoid -performance problems. - -You can use [`git-sizer`](https://github.com/github/git-sizer) to get a snapshot -of repository characteristics and discover problem aspects of your monorepo. - -For example: - -```shell -Processing blobs: 1652370 -Processing trees: 3396199 -Processing commits: 722647 -Matching commits to trees: 722647 -Processing annotated tags: 534 -Processing references: 539 -| Name | Value | Level of concern | -| ---------------------------- | --------- | ------------------------------ | -| Overall repository size | | | -| * Commits | | | -| * Count | 723 k | * | -| * Total size | 525 MiB | ** | -| * Trees | | | -| * Count | 3.40 M | ** | -| * Total size | 9.00 GiB | **** | -| * Total tree entries | 264 M | ***** | -| * Blobs | | | -| * Count | 1.65 M | * | -| * Total size | 55.8 GiB | ***** | -| * Annotated tags | | | -| * Count | 534 | | -| * References | | | -| * Count | 539 | | -| | | | -| Biggest objects | | | -| * Commits | | | -| * Maximum size [1] | 72.7 KiB | * | -| * Maximum parents [2] | 66 | ****** | -| * Trees | | | -| * Maximum entries [3] | 1.68 k | * | -| * Blobs | | | -| * Maximum size [4] | 13.5 MiB | * | -| | | | -| History structure | | | -| * Maximum history depth | 136 k | | -| * Maximum tag depth [5] | 1 | | -| | | | -| Biggest checkouts | | | -| * Number of directories [6] | 4.38 k | ** | -| * Maximum path depth [7] | 13 | * | -| * Maximum path length [8] | 134 B | * | -| * Number of files [9] | 62.3 k | * | -| * Total size of files [9] | 747 MiB | | -| * Number of symlinks [10] | 40 | | -| * Number of submodules | 0 | | -``` - -In this example, a few items are raised with a high level of concern. See the -following sections for information on solving: - -- A high number of references. -- Large blobs. - -### Large number of references - -A reference in Git (a branch or tag) is used to refer to a commit. Each -reference is stored as an individual file. If you are curious, you can go -to any `.git` directory and look under the `refs` directory. - -A large number of references can cause performance problems because, with more references, -object walks that Git does are larger for various operations such as clones, pushes, and -housekeeping tasks. - -#### Mitigation strategies - -To mitigate the effects of a large number of references in a monorepo: - -- Create an automated process for cleaning up old branches. -- If certain references don't need to be visible to the client, hide them using the - [`transfer.hideRefs`](https://git-scm.com/docs/git-config#Documentation/git-config.txt-transferhideRefs) - configuration setting. Because Gitaly ignores any on-server Git configuration, you must change the Gitaly configuration - itself in `/etc/gitlab/gitlab.rb`: - - ```ruby - gitaly['configuration'] = { - # ... - git: { - # ... - config: [ - # ... - { key: "transfer.hideRefs", value: "refs/namespace_to_hide" }, - ], - }, - } - ``` - -In Git 2.42.0 and later, different Git operations can skip over hidden references -when doing an object graph walk. - -### Using LFS for large blobs - -Because Git is built to handle text data, it doesn't handle large -binary files efficiently. - -Therefore, you should store binary or blob files (for example, packages, audio, video, or graphics) -as Large File Storage (LFS) objects. With LFS, the objects are stored externally, such as in Object -Storage, which reduces the number and size of objects in the repository. Storing -objects in external Object Storage can improve performance. - -To analyze if a repository has large objects, you can use a tool like -[`git-sizer`](https://github.com/github/git-sizer) for detailed analysis. This -tool shows details about what makes up the repository, and highlights any areas -of concern. If any large objects are found, you can then remove them with a tool -such as [`git filter-repo`](reducing_the_repo_size_using_git.md). - -For more information, refer to the [Git LFS documentation](../../../topics/git/lfs/index.md). - -## Optimizing large repositories for GitLab - -Other than modifying your workflow and the actual repository, you can take other -steps to maximize performance of monorepos with GitLab. - -### Gitaly pack-objects cache - -For very active repositories with a large number of references and files, consider using the -[Gitaly pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache). -The pack-objects cache: - -- Benefits all repositories on your GitLab server. -- Automatically works for forks. - -You should always: - -- Fetch incrementally. Do not clone in a way that recreates all of the worktree. -- Use shallow clones to reduce data transfer. Be aware that this puts more burden on GitLab instance because of higher CPU impact. - -Control the clone directory if you heavily use a fork-based workflow. Optimize -`git clean` flags to ensure that you remove or keep data that might affect or -speed-up your build. - -For more information, see [Pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache). - -### Reduce concurrent clones in CI/CD - -Large repositories tend to be monorepos. This usually means that these -repositories get a lot of traffic not only from users, but from CI/CD. - -CI/CD loads tend to be concurrent because pipelines are scheduled during set times. -As a result, the Git requests against the repositories can spike notably during -these times and lead to reduced performance for both CI/CD and users alike. - -You should reduce CI/CD pipeline concurrency by staggering them to run at different times. For example, a set running at one time and another set running several -minutes later. - -#### Shallow cloning - -GitLab and GitLab Runner perform a [shallow clone](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) -by default. - -Ideally, you should always use `GIT_DEPTH` with a small number -like 10. This instructs GitLab Runner to perform shallow clones. -Shallow clones make Git request only the latest set of changes for a given branch, -up to desired number of commits as defined by the `GIT_DEPTH` variable. - -This significantly speeds up fetching of changes from Git repositories, -especially if the repository has a very long backlog consisting of a number -of big files because we effectively reduce amount of data transfer. -The following pipeline configuration example makes the runner shallow clone to fetch only a given branch. -The runner does not fetch any other branches nor tags. - -```yaml -variables: - GIT_DEPTH: 10 - -test: - script: - - ls -al -``` - -#### Git strategy - -By default, GitLab is configured to use the [`fetch` Git strategy](../../../ci/runners/configure_runners.md#git-strategy), -which is recommended for large repositories. -This strategy reduces the amount of data to transfer and -does not really impact the operations that you might do on a repository from CI/CD. - -#### Git clone path - -[`GIT_CLONE_PATH`](../../../ci/runners/configure_runners.md#custom-build-directories) allows you to -control where you clone your repositories. This can have implications if you -heavily use big repositories with a fork-based workflow. - -A fork, from the perspective of GitLab Runner, is stored as a separate repository -with a separate worktree. That means that GitLab Runner cannot optimize the usage -of worktrees and you might have to instruct GitLab Runner to use that. - -In such cases, ideally you want to make the GitLab Runner executor be used only -for the given project and not shared across different projects to make this -process more efficient. - -The [`GIT_CLONE_PATH`](../../../ci/runners/configure_runners.md#custom-build-directories) must be -in the directory set in `$CI_BUILDS_DIR`. You can't pick any path from disk. - -#### Git clean flags - -[`GIT_CLEAN_FLAGS`](../../../ci/runners/configure_runners.md#git-clean-flags) allows you to control -whether or not you require the `git clean` command to be executed for each CI/CD -job. By default, GitLab ensures that: - -- You have your worktree on the given SHA. -- Your repository is clean. - -[`GIT_CLEAN_FLAGS`](../../../ci/runners/configure_runners.md#git-clean-flags) is disabled when set -to `none`. On very big repositories, this might be desired because `git -clean` is disk I/O intensive. Controlling that with `GIT_CLEAN_FLAGS: -ffdx --e .build/` (for example) allows you to control and disable removal of some -directories in the worktree between subsequent runs, which can speed-up -the incremental builds. This has the biggest effect if you re-use existing -machines and have an existing worktree that you can re-use for builds. - -For exact parameters accepted by -[`GIT_CLEAN_FLAGS`](../../../ci/runners/configure_runners.md#git-clean-flags), see the documentation -for [`git clean`](https://git-scm.com/docs/git-clean). The available parameters -are dependent on the Git version. - -#### Git fetch extra flags - -[`GIT_FETCH_EXTRA_FLAGS`](../../../ci/runners/configure_runners.md#git-fetch-extra-flags) allows you -to modify `git fetch` behavior by passing extra flags. - -For example, if your project contains a large number of tags that your CI/CD jobs don't rely on, -you could add [`--no-tags`](https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---no-tags) -to the extra flags to make your fetches faster and more compact. - -Also in the case where you repository does _not_ contain a lot of -tags, `--no-tags` can [make a big difference in some cases](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/746). -If your CI/CD builds do not depend on Git tags, setting `--no-tags` is worth trying. - -For more information, see the [`GIT_FETCH_EXTRA_FLAGS` documentation](../../../ci/runners/configure_runners.md#git-fetch-extra-flags). - -#### Fork-based workflow - -Following the guidelines above, let's imagine that we want to: - -- Optimize for a big project (more than 50k files in directory). -- Use forks-based workflow for contributing. -- Reuse existing worktrees. Have preconfigured runners that are pre-cloned with repositories. -- Runner assigned only to project and all forks. - -Let's consider the following two examples, one using `shell` executor and -other using `docker` executor. - -##### `shell` executor example - -Let's assume that you have the following [`config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html). - -```toml -concurrent = 4 - -[[runners]] - url = "GITLAB_URL" - token = "TOKEN" - executor = "shell" - builds_dir = "/builds" - cache_dir = "/cache" - - [runners.custom_build_dir] - enabled = true -``` - -This `config.toml`: - -- Uses the `shell` executor, -- Specifies a custom `/builds` directory where all clones are stored. -- Enables the ability to specify `GIT_CLONE_PATH`, -- Runs at most 4 jobs at once. - -##### `docker` executor example - -Let's assume that you have the following [`config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html). - -```toml -concurrent = 4 - -[[runners]] - url = "GITLAB_URL" - token = "TOKEN" - executor = "docker" - builds_dir = "/builds" - cache_dir = "/cache" - - [runners.docker] - volumes = ["/builds:/builds", "/cache:/cache"] -``` - -This `config.toml`: - -- Uses the `docker` executor, -- Specifies a custom `/builds` directory on disk where all clones are stored. - We host mount the `/builds` directory to make it reusable between subsequent runs - and be allowed to override the cloning strategy. -- Doesn't enable the ability to specify `GIT_CLONE_PATH` as it is enabled by default. -- Runs at most 4 jobs at once. - -##### Our `.gitlab-ci.yml` - -Once we have the executor configured, we need to fine tune our `.gitlab-ci.yml`. - -Our pipeline is most performant if we use the following `.gitlab-ci.yml`: - -```yaml -variables: - GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_CONCURRENT_ID/$CI_PROJECT_NAME - -build: - script: ls -al -``` - -This YAML setting configures a custom clone path. This path makes it possible to re-use worktrees -between the parent project and forks because we use the same clone path for all forks. - -Why use `$CI_CONCURRENT_ID`? The main reason is to ensure that worktrees used are not conflicting -between projects. The `$CI_CONCURRENT_ID` represents a unique identifier within the given executor. -When we use it to construct the path, this directory does not conflict -with other concurrent jobs running. - -### Store custom clone options in `config.toml` - -Ideally, all job-related configuration should be stored in `.gitlab-ci.yml`. -However, sometimes it is desirable to make these schemes part of the runner's configuration. - -In the above example of forks, making this configuration discoverable for users may be preferred, -but this brings administrative overhead as the `.gitlab-ci.yml` needs to be updated for each branch. -In such cases, it might be desirable to keep the `.gitlab-ci.yml` clone path agnostic, but make it -a configuration of the runner. - -We can extend our [`config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -with the following specification that is used by the runner if `.gitlab-ci.yml` does not override it: - -```toml -concurrent = 4 - -[[runners]] - url = "GITLAB_URL" - token = "TOKEN" - executor = "docker" - builds_dir = "/builds" - cache_dir = "/cache" - - environment = [ - "GIT_CLONE_PATH=$CI_BUILDS_DIR/$CI_CONCURRENT_ID/$CI_PROJECT_NAME" - ] - - [runners.docker] - volumes = ["/builds:/builds", "/cache:/cache"] -``` - -This makes the cloning configuration to be part of the given runner -and does not require us to update each `.gitlab-ci.yml`. - -### Reference architectures - -Large repositories tend to be found in larger organisations with many users. The GitLab Quality and Support teams provide several [reference architectures](../../../administration/reference_architectures/index.md) that are the recommended way to deploy GitLab at scale. - -In these types of setups, the GitLab environment used should match a reference architecture to improve performance. - -### Gitaly Cluster - -Gitaly Cluster can notably improve large repository performance because it holds multiple replicas of the repository across several nodes. -As a result, Gitaly Cluster can load balance read requests against those replicas and is fault-tolerant. - -Though Gitaly Cluster is recommended for large repositories, it is a large solution with additional complexity of setup and management. Refer to the -[Gitaly Cluster documentation for more information](../../../administration/gitaly/index.md), specifically the -[Before deploying Gitaly Cluster](../../../administration/gitaly/index.md#before-deploying-gitaly-cluster) section. - -### Keep GitLab up to date - -You should keep GitLab updated to the latest version where possible to benefit from performance improvements and fixes are added continuously to GitLab. +<!-- This redirect file can be deleted after <2023-12-17>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/repository/monorepos/index.md b/doc/user/project/repository/monorepos/index.md new file mode 100644 index 00000000000..144f46cd7d5 --- /dev/null +++ b/doc/user/project/repository/monorepos/index.md @@ -0,0 +1,356 @@ +--- +stage: Systems +group: Gitaly +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Managing monorepos + +Monorepos have become a regular part of development team workflows. While they have many advantages, monorepos can present performance challenges +when using them in GitLab. Therefore, you should know: + +- What repository characteristics can impact performance. +- Some tools and steps to optimize monorepos. + +## Impact on performance + +Because GitLab is a Git-based system, it is subject to similar performance +constraints as Git when it comes to large repositories that are gigabytes in +size. + +Monorepos can be large for [many reasons](https://about.gitlab.com/blog/2022/09/06/speed-up-your-monorepo-workflow-in-git/#characteristics-of-monorepos). + +Large repositories pose a performance risk performance when used in GitLab, especially if a large monorepo receives many clones or pushes a day, which is common for them. + +Git itself has performance limitations when it comes to handling +monorepos. + +Monorepos can also impact notably on hardware, in some cases hitting limitations such as vertical scaling and network or disk bandwidth limits. + +[Gitaly](https://gitlab.com/gitlab-org/gitaly) is our Git storage service built +on top of [Git](https://git-scm.com/). This means that any limitations of +Git are experienced in Gitaly, and in turn by end users of GitLab. + +## Optimize GitLab settings + +You should use as many of the following strategies as possible to minimize +fetches on the Gitaly server. + +### Rationale + +The most resource intensive operation in Git is the +[`git-pack-objects`](https://git-scm.com/docs/git-pack-objects) process. It is +responsible for figuring out all of the commit history and files to send back to +the client. + +The larger the repository, the more commits, files, branches, and tags that a +repository has and the more expensive this operation is. Both memory and CPU +are heavily utilized during this operation. + +Most `git clone` or `git fetch` traffic (which results in starting a `git-pack-objects` process on the server) often come from automated +continuous integration systems such as GitLab CI/CD or other CI/CD systems. +If there is a high amount of such traffic, hitting a Gitaly server with many +clones for a large repository is likely to put the server under significant +strain. + +### Gitaly pack-objects cache + +Turn on the [Gitaly pack-objects cache](../../../../administration/gitaly/configure_gitaly.md#pack-objects-cache), +which reduces the work that the server has to do for clones and fetches. + +#### Rationale + +The [pack-objects cache](../../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) +caches the data that the `git-pack-objects` process produces. This response +is sent back to the Git client initiating the clone or fetch. If several +fetches are requesting the same set of refs, Git on the Gitaly server doesn't have +to re-generate the response data with each clone or fetch call, but instead serves +that data from an in-memory cache that Gitaly maintains. + +This can help immensely in the presence of a high rate of clones for a single +repository. + +For more information, see [Pack-objects cache](../../../../administration/gitaly/configure_gitaly.md#pack-objects-cache). + +### Reduce concurrent clones in CI/CD + +CI/CD loads tend to be concurrent because pipelines are scheduled during set times. +As a result, the Git requests against the repositories can spike notably during +these times and lead to reduced performance for both CI/CD and users alike. + +Reduce CI/CD pipeline concurrency by staggering them to run at different times. +For example, a set running at one time and another set running several minutes +later. + +### Shallow cloning + +In your CI/CD systems, set the +[`--depth`](https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---depthltdepthgt) +option in the `git clone` or `git fetch` call. + +GitLab and GitLab Runner perform a [shallow clone](../../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) +by default. + +If possible, set the clone depth with a small number like 10. Shallow clones make Git request only +the latest set of changes for a given branch, up to desired number of commits. + +This significantly speeds up fetching of changes from Git repositories, +especially if the repository has a very long backlog consisting of a number +of big files because we effectively reduce amount of data transfer. + +The following GitLab CI/CD pipeline configuration example sets the `GIT_DEPTH`. + +```yaml +variables: + GIT_DEPTH: 10 + +test: + script: + - ls -al +``` + +### Git strategy + +Use `git fetch` instead of `git clone` on CI/CD systems if it's possible to keep +a working copy of the repository. + +By default, GitLab is configured to use the [`fetch` Git strategy](../../../../ci/runners/configure_runners.md#git-strategy), +which is recommended for large repositories. + +#### Rationale + +`git clone` gets the entire repository from scratch, whereas `git fetch` only +asks the server for references that do not already exist in the repository. +Naturally, `git fetch` causes the server to do less work. `git-pack-objects` +doesn't have to go through all branches and tags and roll everything up into a +response that gets sent over. Instead, it only has to worry about a subset of +references to pack up. This strategy also reduces the amount of data to transfer. + +### Git clone path + +[`GIT_CLONE_PATH`](../../../../ci/runners/configure_runners.md#custom-build-directories) allows you to +control where you clone your repositories. This can have implications if you +heavily use big repositories with a fork-based workflow. + +A fork, from the perspective of GitLab Runner, is stored as a separate repository +with a separate worktree. That means that GitLab Runner cannot optimize the usage +of worktrees and you might have to instruct GitLab Runner to use that. + +In such cases, ideally you want to make the GitLab Runner executor be used only +for the given project and not shared across different projects to make this +process more efficient. + +The [`GIT_CLONE_PATH`](../../../../ci/runners/configure_runners.md#custom-build-directories) must be +in the directory set in `$CI_BUILDS_DIR`. You can't pick any path from disk. + +### Git clean flags + +[`GIT_CLEAN_FLAGS`](../../../../ci/runners/configure_runners.md#git-clean-flags) allows you to control +whether or not you require the `git clean` command to be executed for each CI/CD +job. By default, GitLab ensures that: + +- You have your worktree on the given SHA. +- Your repository is clean. + +[`GIT_CLEAN_FLAGS`](../../../../ci/runners/configure_runners.md#git-clean-flags) is disabled when set +to `none`. On very big repositories, this might be desired because `git +clean` is disk I/O intensive. Controlling that with `GIT_CLEAN_FLAGS: -ffdx +-e .build/` (for example) allows you to control and disable removal of some +directories in the worktree between subsequent runs, which can speed-up +the incremental builds. This has the biggest effect if you re-use existing +machines and have an existing worktree that you can re-use for builds. + +For exact parameters accepted by +[`GIT_CLEAN_FLAGS`](../../../../ci/runners/configure_runners.md#git-clean-flags), see the documentation +for [`git clean`](https://git-scm.com/docs/git-clean). The available parameters +are dependent on the Git version. + +### Git fetch extra flags + +[`GIT_FETCH_EXTRA_FLAGS`](../../../../ci/runners/configure_runners.md#git-fetch-extra-flags) allows you +to modify `git fetch` behavior by passing extra flags. + +For example, if your project contains a large number of tags that your CI/CD jobs don't rely on, +you could add [`--no-tags`](https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---no-tags) +to the extra flags to make your fetches faster and more compact. + +Also in the case where you repository does _not_ contain a lot of +tags, `--no-tags` can [make a big difference in some cases](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/746). +If your CI/CD builds do not depend on Git tags, setting `--no-tags` is worth trying. + +For more information, see the [`GIT_FETCH_EXTRA_FLAGS` documentation](../../../../ci/runners/configure_runners.md#git-fetch-extra-flags). + +### Configure Gitaly negotiation timeouts + +You might experience a `fatal: the remote end hung up unexpectedly` error when attempting to fetch or archive: + +- Large repositories. +- Many repositories in parallel. +- The same large repository in parallel. + +You can attempt to mitigate this issue by increasing the default negotiation timeout values. For more information, see +[Configure negotiation timeouts](../../../../administration/gitaly/configure_gitaly.md#configure-negotiation-timeouts). + +## Optimize your repository + +Another avenue to keeping GitLab scalable with your monorepo is to optimize the +repository itself. + +### Profiling repositories + +Large repositories generally experience performance issues in Git. Knowing why +your repository is large can help you develop mitigation strategies to avoid +performance problems. + +You can use [`git-sizer`](https://github.com/github/git-sizer) to get a snapshot +of repository characteristics and discover problem aspects of your monorepo. + +For example: + +```shell +Processing blobs: 1652370 +Processing trees: 3396199 +Processing commits: 722647 +Matching commits to trees: 722647 +Processing annotated tags: 534 +Processing references: 539 +| Name | Value | Level of concern | +| ---------------------------- | --------- | ------------------------------ | +| Overall repository size | | | +| * Commits | | | +| * Count | 723 k | * | +| * Total size | 525 MiB | ** | +| * Trees | | | +| * Count | 3.40 M | ** | +| * Total size | 9.00 GiB | **** | +| * Total tree entries | 264 M | ***** | +| * Blobs | | | +| * Count | 1.65 M | * | +| * Total size | 55.8 GiB | ***** | +| * Annotated tags | | | +| * Count | 534 | | +| * References | | | +| * Count | 539 | | +| | | | +| Biggest objects | | | +| * Commits | | | +| * Maximum size [1] | 72.7 KiB | * | +| * Maximum parents [2] | 66 | ****** | +| * Trees | | | +| * Maximum entries [3] | 1.68 k | * | +| * Blobs | | | +| * Maximum size [4] | 13.5 MiB | * | +| | | | +| History structure | | | +| * Maximum history depth | 136 k | | +| * Maximum tag depth [5] | 1 | | +| | | | +| Biggest checkouts | | | +| * Number of directories [6] | 4.38 k | ** | +| * Maximum path depth [7] | 13 | * | +| * Maximum path length [8] | 134 B | * | +| * Number of files [9] | 62.3 k | * | +| * Total size of files [9] | 747 MiB | | +| * Number of symlinks [10] | 40 | | +| * Number of submodules | 0 | | +``` + +In this example, a few items are raised with a high level of concern. See the +following sections for information on solving: + +- A large number of references. +- Large blobs. + +### Large number of references + +[References in Git](https://git-scm.com/book/en/v2/Git-Internals-Git-References) +are branch and tag names that point to a particular commit. You can use the `git +for-each-ref` command to list all references present in a repository. A large +number of references in a repository can have detrimental impact on the command's +performance. To understand why, we need to understand how Git stores references +and uses them. + +In general, Git stores all references as loose files in the `.git/refs` folder of +the repository. As the number of references grows, the seek time to find a +particular reference in the folder also increases. Therefore, every time Git has +to parse a reference, there is an increased latency due to the added seek time +of the file system. + +To resolve this issue, Git uses [pack-refs](https://git-scm.com/docs/git-pack-refs). In short, instead of storing each +reference in a single file, Git creates a single `.git/packed-refs` file that +contains all the references for that repository. This file reduces storage space +while also increasing performance because seeking within a single file is faster +than seeking a file within a directory. However, creating and updating new references +is still done through loose files and are not added to the `packed-refs` file. To +recreate the `packed-refs` file, run `git pack-refs`. + +Gitaly runs `git pack-refs` during [housekeeping](../../../../administration/housekeeping.md#heuristical-housekeeping) +to move loose references into `packed-refs` files. While this is very beneficial +for most repositories, write-heavy repositories still have the problem that: + +- Creating or updating references creates new loose files. +- Deleting references involves modifying the existing `packed-refs` file + altogether to remove the existing reference. + +These problems still cause the same performance issues. + +In addition, fetches and clones from repositories includes the transfer +of missing objects from the server to the client. When there are numerous +references, Git iterates over all references and walks the internal graph +structure for each reference to find the missing objects to transfer to +the client. Iteration and walking are CPU-intensive operations that increase +the latency of these commands. + +In repositories with a lot of activity, this often causes a domino effect because +every operation is slower and each operation stalls subsequent operations. + +#### Mitigation strategies + +To mitigate the effects of a large number of references in a monorepo: + +- Create an automated process for cleaning up old branches. +- If certain references don't need to be visible to the client, hide them using the + [`transfer.hideRefs`](https://git-scm.com/docs/git-config#Documentation/git-config.txt-transferhideRefs) + configuration setting. Because Gitaly ignores any on-server Git configuration, you must change the Gitaly configuration + itself in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitaly['configuration'] = { + # ... + git: { + # ... + config: [ + # ... + { key: "transfer.hideRefs", value: "refs/namespace_to_hide" }, + ], + }, + } + ``` + +In Git 2.42.0 and later, different Git operations can skip over hidden references +when doing an object graph walk. + +### Large blobs + +The presence of large files (called blobs in Git), can be problematic for Git +because it does not handle large binary files efficiently. If there are blobs over +10 MB or instance in the `git-sizer` output, this probably means there is binary +data in your repository. + +#### Use LFS for large blobs + +Store binary or blob files (for example, packages, audio, video, or graphics) +as Large File Storage (LFS) objects. With LFS, the objects are stored externally, such as in Object +Storage, which reduces the number and size of objects in the repository. Storing +objects in external Object Storage can improve performance. + +For more information, refer to the [Git LFS documentation](../../../../topics/git/lfs/index.md). + +### Reference architectures + +Large repositories tend to be found in larger organisations with many users. The +GitLab Quality Engineering and Support teams provide several [reference architectures](../../../../administration/reference_architectures/index.md) that +are the recommended way to deploy GitLab at scale. + +In these types of setups, the GitLab environment used should match a reference +architecture to improve performance. diff --git a/doc/user/project/repository/monorepos/observability.md b/doc/user/project/repository/monorepos/observability.md new file mode 100644 index 00000000000..a54b4bef9d5 --- /dev/null +++ b/doc/user/project/repository/monorepos/observability.md @@ -0,0 +1,176 @@ +--- +stage: Systems +group: Gitaly +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Metrics for measuring monorepo performance + +The following metrics can be used to measure server side performance of your +monorepo. These metrics are not limited to monorepo performance and are more +general metrics to measure Gitaly performance, but they are especially relevant +when running a monorepo. + +## Clones and Fetches + +The most frequent expensive operation are clones and fetches. When taken as a +percentage of system resources consumed, these operations often contribute to +90% or more of system resources on Gitaly nodes. Here are some logs and metrics +that can provide useful signals. + +### CPU and Memory + +There are two main RPCs that handle clones/fetches. The following log entry +fields an be used to inspect how much system resources are consumed by +clones/fetches for a given repository. + +The following are log entry fields in the Gitaly logs that can be filtered on: + +| Log field | Values to filter on | Why? | +|------------------|---------------------|-----------------------------------------------------------------------------------------------| +| `json.grpc.method` | `PostReceivePack` | This is the RPC that handles HTTP clones/fetches | +| `json.grpc.method` | SSHReceivePack | This is the RPC that handles SSH clones/fetches | +| `json.grpc.code` | OK | Indicates the RPC has successfully served its request | +| `json.grpc.code` | Canceled | Often times indicates the client killed the connection, usually due to a timeout of some sort | +| `json.grpc.code` | ResourceExhausted | Indicates there are too many Git processes being spawned on the machine simultaneously | +| `json.user_id` | A `user_id` who initiated the clone/fetch. This is in the form of `user-<user_id>` eg: `user-22345` | Indicates there are too many Git processes being spawned on the machine simultaneously | +| `json.username` | A username who initiated the clone/fetch. eg: `ilovecoding` | In order to see how many clones/fetches were from a given user. This is sometimes helpful to find excessive clone operations by a single user | +| `json.grpc.request.glRepository` | A repository in question. In the form of `project-<project_id>` eg: `project-214` | In order to see how many clones/fetches were for a given repository. | +| `json.grpc.request.glProjectPath` | A repository in question. In the form of a project path eg: `my-org/coolproject` | In order to see how many clones/fetches were for a given repository. | + +The following are log entry fields that give useful information about cpu and +memory: + +| Log field to inspect | What does it tell you? | +|--------------------------|-----------------------------------------------------------------| +| `json.command.cpu_time_ms` | How much CPU time used by subprocesses this RPC spawned | +| `json.command.maxrss` | How much memory was consumed from subprocesses this RPC spawned | + +Example log message: + +```json +{ + "command.count":2, + "command.cpu_time_ms":420, + "command.inblock":0, + "command.majflt":0, + "command.maxrss":3342152, + "command.minflt":24316, + "command.oublock":56, + "command.real_time_ms":626, + "command.spawn_token_fork_ms":4, + "command.spawn_token_wait_ms":0, + "command.system_time_ms":172, + "command.user_time_ms":248, + "component":"gitaly.StreamServerInterceptor", + "correlation_id":"20HCB3DAEPLV08UGNIYT9HJ4JW", + "environment":"gprd", + "feature_flags":"", + "fqdn":"file-99-stor-gprd.c.gitlab-production.internal", + "grpc.code":"OK", + "grpc.meta.auth_version":"v2", + "grpc.meta.client_name":"gitlab-workhorse", + "grpc.meta.deadline_type":"none", + "grpc.meta.method_operation":"mutator", + "grpc.meta.method_scope":"repository", + "grpc.meta.method_type":"bidi_stream", + "grpc.method":"PostReceivePack", + "grpc.request.fullMethod":"/gitaly.SmartHTTPService/PostReceivePack", + "grpc.request.glProjectPath":"r2414/revenir/development/machinelearning/protein-ddg", + "grpc.request.glRepository":"project-47506374", + "grpc.request.payload_bytes":911, + "grpc.request.repoPath":"@hashed/db/ab/dbabf83f57affedc9a001dc6c6f6b47bb431bd47d7254edd1daf24f0c38793a9.git", + "grpc.request.repoStorage":"nfs-file99", + "grpc.response.payload_bytes":54 + "grpc.service":"gitaly.SmartHTTPService", + "grpc.start_time":"2023-10-16T20:40:08.836", + "grpc.time_ms":631.486, + "hostname":"file-99-stor-gprd", + "level":"info", + "msg":"finished streaming call with code OK", + "pid":1741362, + "remote_ip":"108.163.136.48", + "shard":"default", + "span.kind":"server", + "stage":"main", + "system":"grpc", + "tag":"gitaly", + "tier":"stor", + "time":"2023-10-16T20:40:09.467Z", + "trace.traceid":"AAB3QAeD8G+H9VNmzOi2CztMAcJv1+g4+l1cAgA=", + "type":"gitaly", + "user_id":"user-14857500", + "username":"ctx_ckottke", + } +``` + +### Read distribution + +The `gitaly_praefect_read_distribution` Prometheus metric is a +[counter](https://prometheus.io/docs/concepts/metric_types/#counter) that +indicates how many reads have gone to which Gitaly nodes. This metric has two +vectors: + +| Metric Name | Vector | What is it? | +|-------------------------------------|------------------------------------------------------------------------------------------------------------------------| +| `gitaly_praefect_read_distribution` | `virtual_storage`| The [virtual storage](../../../../administration/gitaly/praefect.md) name | +| `gitaly_praefect_read_distribution` | `storage` | The Gitaly storage name | + +### Pack objects cache + +The [pack objects cache](../../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) +can be observed through both logs as well as Prometheus metrics. + +| Log field name | Description | +|:---|:---| +| `pack_objects_cache.hit` | Indicates whether the current pack-objects cache was hit (`true` or `false`) | +| `pack_objects_cache.key` | Cache key used for the pack-objects cache | +| `pack_objects_cache.generated_bytes` | Size (in bytes) of the new cache being written | +| `pack_objects_cache.served_bytes` | Size (in bytes) of the cache being served | +| `pack_objects.compression_statistics` | Statistics regarding pack-objects generation | +| `pack_objects.enumerate_objects_ms` | Total time (in ms) spent enumerating objects sent by clients | +| `pack_objects.prepare_pack_ms` | Total time (in ms) spent preparing the packfile before sending it back to the client | +| `pack_objects.write_pack_file_ms` | Total time (in ms) spent sending back the packfile to the client. Highly dependent on the client's internet connection | +| `pack_objects.written_object_count` | Total number of objects Gitaly sends back to the client | + +Example log message: + +```json +{ +"bytes":26186490, +"correlation_id":"01F1MY8JXC3FZN14JBG1H42G9F", +"grpc.meta.deadline_type":"none", +"grpc.method":"PackObjectsHook", +"grpc.request.fullMethod":"/gitaly.HookService/PackObjectsHook", +"grpc.request.glProjectPath":"root/gitlab-workhorse", +"grpc.request.glRepository":"project-2", +"grpc.request.repoPath":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git", +"grpc.request.repoStorage":"default", +"grpc.request.topLevelGroup":"@hashed", +"grpc.service":"gitaly.HookService", +"grpc.start_time":"2021-03-25T14:57:52.747Z", +"level":"info", +"msg":"finished unary call with code OK", +"peer.address":"@", +"pid":20961, +"span.kind":"server", +"system":"grpc", +"time":"2021-03-25T14:57:53.543Z", +"pack_objects.compression_statistics": "Total 145991 (delta 68), reused 6 (delta 2), pack-reused 145911", +"pack_objects.enumerate_objects_ms": 170, +"pack_objects.prepare_pack_ms": 7, +"pack_objects.write_pack_file_ms": 786, +"pack_objects.written_object_count": 145991, +"pack_objects_cache.generated_bytes": 49533030, +"pack_objects_cache.hit": "false", +"pack_objects_cache.key": "123456789", +"pack_objects_cache.served_bytes": 49533030, +"peer.address": "127.0.0.1", +"pid": 8813, +} +``` + +| Prometheus metric name | Vector | Description | +|:---|:---| +| `gitaly_pack_objects_served_bytes_total` | | Size (in bytes) of the cache being served| +| `gitaly_pack_objects_cache_lookups_total` | `result` | `hit` or `miss`,indicating whether or not a cache lookup resulted in a cache hit or miss | diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index bfe8964a876..ff9ef5b78f8 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -207,8 +207,8 @@ This: - Removes any internal Git references to old commits. - Runs `git gc --prune=30.minutes.ago` against the repository to remove unreferenced objects. Repacking your repository temporarily - causes the size of your repository to increase significantly, because the old pack files are not removed until the - new pack files have been created. + causes the size of your repository to increase significantly, because the old packfiles are not removed until the + new packfiles have been created. - Unlinks any unused LFS objects attached to your project, freeing up storage space. - Recalculates the size of your repository on disk. @@ -324,3 +324,33 @@ are accurate. To expedite this process, see the ['Prune Unreachable Objects' housekeeping task](../../../administration/housekeeping.md). + +### Sidekiq process fails to export a project + +Occasionally the Sidekiq process can fail to export a project, for example if +it is terminated during execution. + +To bypass the Sidekiq process, use the Rails console to manually trigger the project export: + +```ruby +project = Project.find(1) +current_user = User.find_by(username: 'my-user-name') +RequestStore.begin! +ActiveRecord::Base.logger = Logger.new(STDOUT) +params = {} + +::Projects::ImportExport::ExportService.new(project, current_user, params).execute(nil) +``` + +This makes the export available through the UI, but does not trigger an email to the user. +To manually trigger the project export and send an email: + +```ruby +project = Project.find(1) +current_user = User.find_by(username: 'my-user-name') +RequestStore.begin! +ActiveRecord::Base.logger = Logger.new(STDOUT) +params = {} + +ProjectExportWorker.new.perform(current_user.id, project.id) +``` diff --git a/doc/user/project/repository/signed_commits/ssh.md b/doc/user/project/repository/signed_commits/ssh.md index 3572e56da84..c87a992fdac 100644 --- a/doc/user/project/repository/signed_commits/ssh.md +++ b/doc/user/project/repository/signed_commits/ssh.md @@ -48,12 +48,12 @@ To configure Git to use your key: git config --global gpg.format ssh ``` -1. Specify which SSH key should be used as the signing key, changing the filename - (here, `~/.ssh/examplekey`) to the location of your key. The filename may +1. Specify which public SSH key to use as the signing key and change the filename + (`~/.ssh/examplekey.pub`) to the location of your key. The filename might differ, depending on how you generated your key: ```shell - git config --global user.signingkey ~/.ssh/examplekey + git config --global user.signingkey ~/.ssh/examplekey.pub ``` ## Sign commits with your SSH key diff --git a/doc/user/project/service_desk/configure.md b/doc/user/project/service_desk/configure.md index f8f4ab44e5a..172a105cc28 100644 --- a/doc/user/project/service_desk/configure.md +++ b/doc/user/project/service_desk/configure.md @@ -272,6 +272,89 @@ External participants can [reply by email](../../../administration/reply_by_emai GitLab uses an email reply address with a 32-character reply key that corresponds to the ticket. When a custom email is configured, GitLab generates the reply address from that email. +### Use Google Workspace with your own domain + +Set up a custom email address for Service Desk when using Google Workspace with your own domain. + +Prerequisites: + +- You already have a Google Workspace account. +- You can create new accounts for your tenant. + +To configure a custom Service Desk email address with Google Workspace: + +1. [Configure a Google Workspace account](#configure-a-google-workspace-account). +1. [Configure email forwarding](#configure-email-forwarding). +1. [Configure custom email address](#configure-custom-email-address). + +#### Configure a Google Workspace account + +First, you must create and configure a Google Workspace account. + +In Google Workspace: + +1. Create a new account for the custom email address you'd like to use (for example, `support@example.com`). +1. Sign in to that account and activate + [two-factor authentication](https://myaccount.google.com/u/3/signinoptions/two-step-verification). +1. [Create an app password](https://myaccount.google.com/u/3/apppasswords) that you can use as your + SMTP password. + Store it in a secure place and remove spaces between the characters. + +Next, you must [configure email forwarding](#configure-email-forwarding). + +#### Configure email forwarding + +The following steps require moving between GitLab and Google Workspace. + +In GitLab: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > General** +1. Expand **Service Desk**. +1. Note the email address below **Service Desk email address to forward emails to**. + +In Google Workspace: + +1. Sign in to the custom email account and open the [Forwarding and POP/IMAP](https://mail.google.com/mail/u/0/#settings/fwdandpop) settings page. +1. Select **Add a forwarding address**. +1. Enter the Service Desk address from the custom email form. +1. Select **Next**. +1. Confirm your input and select **Proceed**. Google sends an email to the Service Desk address and + requires a confirmation code. + +In GitLab: + +1. Go to **Issues** of the project and wait for a new issue to be created from the confirmation + email from Google. +1. Open the issue and note the confirmation code. +1. (Optional) Delete the issue. + +In Google Workspace: + +1. Enter the confirmation code and select **Verify**. +1. Select **Forward a copy of incoming mail to** and make sure the Service Desk address is selected + from the dropdown list. +1. At the bottom of the page, select **Save Changes**. + +Next, [configure a custom email address](#configure-a-custom-email-address) to use with Service Desk. + +#### Configure custom email address + +In GitLab: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > General** +1. Expand **Service Desk** and find the custom email settings. +1. Complete the fields: + - **Custom email address**: Your custom email address. + - **SMTP host**: `smtp.gmail.com`. + - **SMTP port**: `587`. + - **SMTP username**: Prefilled with the custom email address. + - **SMTP password**: The app password you previously created for the custom email account. +1. Select **Save and test connection** +1. After the [verification process](#verification) you should be able to + [enable the custom email address](#enable-or-disable-the-custom-email-address). + ### Known issues - Some service providers don't allow SMTP connections any more. diff --git a/doc/user/project/service_desk/using_service_desk.md b/doc/user/project/service_desk/using_service_desk.md index 73d85d418d9..ad97a36bbb0 100644 --- a/doc/user/project/service_desk/using_service_desk.md +++ b/doc/user/project/service_desk/using_service_desk.md @@ -70,12 +70,12 @@ To view Service Desk issues: #### Redesigned issue list -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/413092) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `service_desk_vue_list`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/413092) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `service_desk_vue_list`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/413092) in GitLab 16.5. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `service_desk_vue_list`. -On GitLab.com, this feature is not available. -The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `service_desk_vue_list`. +On GitLab.com, this feature is available. When this feature is enabled, the Service Desk issue list more closely matches the regular issue list. Available features include: diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index d9c114c0a59..623c61744f7 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -100,7 +100,6 @@ When you disable a feature, the following additional features are also disabled: - **Merge requests** - **CI/CD** - - **Container Registry** - **Git Large File Storage** - **Packages** @@ -253,6 +252,10 @@ You can also [delete projects using the Rails console](../working_with_projects. > - [Disabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95495) in GitLab 15.3. > - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. +Prerequisite: + +- You must have the Owner role for the project. + Projects in a group (not a personal namespace) can be deleted after a delay period. On self-managed instances, group administrators can define a deletion delay period of between 1 and 90 days. @@ -267,16 +270,16 @@ and use the Rails console to > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/191367) in GitLab 14.1. > - Option to delete projects immediately from the Admin Area and as a group setting removed [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. +Prerequisites: + +- You must have the Owner role for the project. +- The project must be [marked for deletion](#delete-a-project). + If you don't want to wait for delayed deletion, you can delete a project immediately. To do this, perform the steps for [deleting a projects](#delete-a-project) again. In the first cycle of deleting a project, the project is moved to the delayed deletion queue and automatically deleted after the retention period has passed. If during this delayed deletion time you run a second deletion cycle, the project is deleted immediately. -Prerequisites: - -- You must have the Owner role for a project. -- You have [marked the project for deletion](#delete-a-project). - To immediately delete a project marked for deletion: 1. On the left sidebar, select **Search or go to** and find your project. @@ -287,7 +290,10 @@ To immediately delete a project marked for deletion: ## Restore a project **(PREMIUM ALL)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6. +Prerequisites: + +- You must have the Owner role for the project. +- The project must be [marked for deletion](#delete-a-project). To restore a project marked for deletion: diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 29d57328532..7de8a7beab5 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, howto --- @@ -79,7 +79,8 @@ To revoke a project access token: ## Scopes for a project access token -> `k8s_proxy` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422408) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `k8s_proxy_pat`. Enabled by default. +> - `k8s_proxy` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/422408) in GitLab 16.4 [with a flag](../../../administration/feature_flags.md) named `k8s_proxy_pat`. Enabled by default. +> - Feature flag `k8s_proxy_pat` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/131518) in GitLab 16.5. The scope determines the actions you can perform when you authenticate with a project access token. diff --git a/doc/user/project/system_notes.md b/doc/user/project/system_notes.md index 661f10290c6..73509846990 100644 --- a/doc/user/project/system_notes.md +++ b/doc/user/project/system_notes.md @@ -51,6 +51,15 @@ The filtering options are: 1. Go to **Activity**. 1. For **Sort or filter**, select **Show all activity**. +## Privacy considerations + +You can see only the system notes linked to objects you can access. + +For example, if someone mentions your issue 111 in an issue in their private project: + +- The project members see the following note in issue 111: `Alex Garcia mentioned in agarcia/private-project#222`. +- Non-members of the project can't see the note at all. + ## Related topics - [Notes API](../../api/notes.md) diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 4f3aa0d0d49..a80c699eab7 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -177,6 +177,25 @@ You need at least the Developer role to move a wiki page: change the **Title** from `about` to `/about`. 1. Select **Save changes**. +## Export a wiki page + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414691) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `print_wiki`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134251/) in GitLab 16.5. + +FLAG: +On self-managed GitLab, by default this feature is available. +To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `print_wiki`. +On GitLab.com, this feature is available. + +You can export a wiki page as a PDF file: + +1. On the left sidebar, select **Search or go to** and find your project or group. +1. Select **Plan > Wiki**. +1. Go to the page you want to export. +1. Select the vertical ellipsis (**{ellipsis_v}**), and then select **Print as PDF**. + +A PDF of the wiki page is created. + ## View history of a wiki page The changes of a wiki page over time are recorded in the wiki's Git repository. @@ -389,7 +408,7 @@ To clear all data from a project wiki and recreate it in a blank state: p = Project.find_by_full_path('<username-or-group>/<project-name>') # This command deletes the wiki project from the filesystem. - GitlabShellWorker.perform_in(0, :remove_repository, p.repository_storage, p.wiki.disk_path) + p.wiki.repository.remove # Refresh the wiki repository state. p.wiki.repository.expire_exists_cache diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 75eed6d4b52..b9c64739de0 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -20,11 +20,12 @@ under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists ## Limitations on project and group names - Project or group names must start with a letter, digit, emoji, or "_". -- Project or group names can only contain letters, digits, emoji, "_", ".", "+", dashes, or spaces. +- Project names can only contain letters, digits, emoji, "_", ".", "+", dashes, or spaces. +- Group names can only contain letters, digits, emoji, "_", ".", parenthesis, dashes, or spaces. - Project or group slugs must start with a letter or digit. -- Project or group slugs can only contain letters, digits, '_', '.', '+', or dashes. +- Project or group slugs can only contain letters, digits, '_', '.', or dashes. - Project or group slugs must not contain consecutive special characters. -- Project or group slugs cannot end with a special character. +- Project or group slugs cannot start or end with a special character. - Project or group slugs cannot end in `.git` or `.atom`. ## Reserved project names diff --git a/doc/user/rich_text_editor.md b/doc/user/rich_text_editor.md index c60c89eb0de..fe3ac56b79c 100644 --- a/doc/user/rich_text_editor.md +++ b/doc/user/rich_text_editor.md @@ -12,15 +12,7 @@ type: index, reference > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382636) for [discussions](discussions/index.md), and creating and editing issues and merge requests in GitLab 15.11 with the same flag. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407507) for epics in GitLab 16.1 with the same flag. > - Feature flag `content_editor_on_issues` enabled by default in GitLab 16.2. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, an administrator -can [disable the feature flag](../administration/feature_flags.md) named `content_editor_on_issues`. -On GitLab.com, this feature is available. - -The rich text editor is a "what you see is what you get" (WYSIWYG) editor so you can use -[GitLab Flavored Markdown](markdown.md) in descriptions and comments, even if you can't remember all -of its syntax. +> - Feature flag `content_editor_on_issues` removed in GitLab 16.5. ![Rich text editor in GitLab](img/rich_text_editor_01_v16_2.png) diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 8c7db5ca29e..e8dfbfa675a 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -24,6 +24,7 @@ by disabling one or more [`ops` feature flags](../../development/feature_flags/i |----------------|------------------------------------|-------------------------------------------------------------------------------------------| | Code | `global_search_code_tab` | When enabled, global search includes code. | | Commits | `global_search_commits_tab` | When enabled, global search includes commits. | +| Epics | `global_search_epics_tab` | When enabled, global search includes epics. | | Issues | `global_search_issues_tab` | When enabled, global search includes issues. | | Merge requests | `global_search_merge_requests_tab` | When enabled, global search includes merge requests. | | Users | `global_search_users_tab` | When enabled, global search includes users. | diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index b0ef1fcc99a..fa03cb54ba3 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -1,6 +1,6 @@ --- -stage: Create -group: IDE +stage: none +group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/user/ssh.md b/doc/user/ssh.md index 0e10fea18ad..482c473e285 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -1,6 +1,6 @@ --- stage: Govern -group: Authentication and Authorization +group: Authentication info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/storage_management_automation.md b/doc/user/storage_management_automation.md index 9a505d23597..96f9ecd11a8 100644 --- a/doc/user/storage_management_automation.md +++ b/doc/user/storage_management_automation.md @@ -5,13 +5,14 @@ group: Utilization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Storage management automation **(FREE ALL)** +# Automate storage management **(FREE ALL)** -You can manage your storage through the GitLab UI and the API. This page describes how to -automate storage analysis and cleanup to manage your [usage quota](usage_quotas.md). You can also -manage your storage usage by making your pipelines more efficient. For more information, see [pipeline efficiency](../ci/pipelines/pipeline_efficiency.md). +This page describes how to automate storage analysis and cleanup to manage your storage usage +with the GitLab REST API. -You can also use the [GitLab community forum and Discord](https://about.gitlab.com/community/) to ask for help with API automation. +You can also manage your storage usage by improving [pipeline efficiency](../ci/pipelines/pipeline_efficiency.md). + +For more help with API automation, you can also use the [GitLab community forum and Discord](https://about.gitlab.com/community/). ## API requirements @@ -19,7 +20,7 @@ To automate storage management, your GitLab.com SaaS or self-managed instance mu ### API authentication scope -You must use the following scopes to [authenticate](../api/rest/index.md#authentication) with the API: +Use the following scopes to [authenticate](../api/rest/index.md#authentication) with the API: - Storage analysis: - Read API access with the `read_api` scope. @@ -30,15 +31,20 @@ You must use the following scopes to [authenticate](../api/rest/index.md#authent You can use command-line tools or a programming language to interact with the REST API. -### Command line +### Command line tools + +To send API requests, install either: + +- curl with your preferred package manager. +- [GitLab CLI](../editor_extensions/gitlab_cli/index.md) and use the `glab api` subcommand. -You must install the following tools to send API requests: +To format JSON responses, install `jq`. For more information, see [Tips for productive DevOps workflows: JSON formatting with jq and CI/CD linting automation](https://about.gitlab.com/blog/2021/04/21/devops-workflows-json-format-jq-ci-cd-lint/). -- Install `curl` with your preferred package manager. -- Install the [GitLab CLI](../editor_extensions/gitlab_cli/index.md) and use the `api` subcommand. -- Install `jq` to format JSON responses. For more information, see [Tips for productive DevOps workflows: JSON formatting with jq and CI/CD linting automation](https://about.gitlab.com/blog/2021/04/21/devops-workflows-json-format-jq-ci-cd-lint/). +To use these tools with the REST API: -Example with `curl` and `jq`: +::Tabs + +:::TabTitle curl ```shell export GITLAB_TOKEN=xxx @@ -46,7 +52,7 @@ export GITLAB_TOKEN=xxx curl --silent --header "Authorization: Bearer $GITLAB_TOKEN" "https://gitlab.com/api/v4/user" | jq ``` -Example with the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): +:::TabTitle GitLab CLI ```shell glab auth login @@ -54,18 +60,25 @@ glab auth login glab api groups/YOURGROUPNAME/projects ``` +::EndTabs + #### Using the GitLab CLI -Some API endpoints require [pagination](../api/rest/index.md#pagination) and subsequent page fetches to retrieve all results. The [GitLab CLI](../editor_extensions/gitlab_cli/index.md) provides the flag `--paginate`. +Some API endpoints require [pagination](../api/rest/index.md#pagination) and subsequent page fetches to retrieve all results. The GitLab CLI provides the flag `--paginate`. -Requests that require sending a POST body formatted as JSON data can be written as `key=value` pairs passed to the `--raw-field` parameter. +Requests that require a POST body formatted as JSON data can be written as `key=value` pairs passed to the `--raw-field` parameter. For more information, see the [GitLab CLI endpoint documentation](../editor_extensions/gitlab_cli/index.md#core-commands). ### API client libraries -The storage management and cleanup automation methods described in this page use the [`python-gitlab`](https://python-gitlab.readthedocs.io/en/stable/) library in programmatic example. The `python-gitlab` library provides -a feature-rich programming interface. For more information about use cases for the `python-gitlab` library, +The storage management and cleanup automation methods described in this page use: + +- The [`python-gitlab`](https://python-gitlab.readthedocs.io/en/stable/) library, which provides +a feature-rich programming interface. +- The `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` script in the [GitLab API with Python](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/) project. + +For more information about use cases for the `python-gitlab` library, see [Efficient DevSecOps workflows: Hands-on `python-gitlab` API automation](https://about.gitlab.com/blog/2023/02/01/efficient-devsecops-workflows-hands-on-python-gitlab-api-automation/). For more information about other API client libraries, see [Third-party clients](../api/rest/index.md#third-party-clients). @@ -73,9 +86,9 @@ For more information about other API client libraries, see [Third-party clients] NOTE: Use [GitLab Duo Code Suggestions](project/repository/code_suggestions/index.md) to write code more efficiently. -## Strategies for storage analysis +## Storage analysis -### Identify the storage types +### Identify storage types The [projects API endpoint](../api/projects.md#list-all-projects) provides statistics for projects in your GitLab instance. To use the projects API endpoint, set the `statistics` key to boolean `true`. @@ -90,9 +103,11 @@ This data provides insight into storage consumption of the project by the follow - `uploads_size`: Uploads storage - `wiki_size`: Wiki storage -Additional queries are required for detailed storage statistics for [job artifacts](../api/job_artifacts.md), the [container registry](../api/container_registry.md), the [package registry](../api/packages.md) and [dependency proxy](../api/dependency_proxy.md). It is explained later in this how-to. +To identify storage types: -Example that uses `curl` and `jq` on the command line: +::Tabs + +:::TabTitle curl ```shell curl --silent --header "Authorization: Bearer $GITLAB_TOKEN" "https://gitlab.com/api/v4/projects/$GL_PROJECT_ID?statistics=true" | jq --compact-output '.id,.statistics' | jq @@ -111,7 +126,7 @@ curl --silent --header "Authorization: Bearer $GITLAB_TOKEN" "https://gitlab.com } ``` -Example that uses the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): +:::TabTitle GitLab CLI ```shell export GL_PROJECT_ID=48349590 @@ -131,7 +146,7 @@ glab api --method GET projects/$GL_PROJECT_ID --field 'statistics=true' | jq --c } ``` -Example using the `python-gitlab` library: +:::TabTitle Python ```python project_obj = gl.projects.get(project.id, statistics=True) @@ -139,7 +154,9 @@ project_obj = gl.projects.get(project.id, statistics=True) print("Project {n} statistics: {s}".format(n=project_obj.name_with_namespace, s=json.dump(project_obj.statistics, indent=4))) ``` -You can find an example implementation in the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` which is located in the [GitLab API with Python project](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/). Export the `GL_GROUP_ID` environment variable and run the script to see the project statistics printed in the terminal. +::EndTabs + +To print statistics for the project to the terminal, export the `GL_GROUP_ID` environment variable and run the script: ```shell export GL_TOKEN=xxx @@ -162,12 +179,12 @@ Project Developer Evangelism and Technical Marketing at GitLab / playground / A } ``` -### Analyzing multiple subgroups and projects +### Analyze storage in projects and groups -You can use automation to analyze multiple projects and groups. For example, you can start at the top namespace level, +You can automate analysis of multiple projects and groups. For example, you can start at the top namespace level, and recursively analyze all subgroups and projects. You can also analyze different storage types. -Here's an example of an algorithm that analyzes multiple subgroups and projects: +Here's an example of an algorithm to analyze multiple subgroups and projects: 1. Fetch the top-level namespace ID. You can copy the ID value from the [namespace/group overview](../user/namespace/index.md#types-of-namespaces). 1. Fetch all [subgroups](../api/groups.md#list-a-groups-subgroups) from the top-level group, and save the IDs in a list. @@ -175,7 +192,17 @@ Here's an example of an algorithm that analyzes multiple subgroups and projects: 1. Identify the storage type to analyze, and collect the information from project attributes, like project statistics, and job artifacts. 1. Print an overview of all projects, grouped by group, and their storage information. -Example with the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): +The shell approach with `glab` might be more suitable for smaller analyses. For larger analyses, you should use a script that +uses the API client libraries. This type of script can improve readability, data storage, flow control, testing, and reusability. + +To ensure the script doesn't reach [API rate limits](../api/rest/index.md#rate-limits), the following +example code is not optimized for parallel API requests. + +To implement this algorithm: + +::Tabs + +:::TabTitle GitLab CLI ```shell export GROUP_NAME="gitlab-de" @@ -221,10 +248,7 @@ glab api projects/48349590/jobs | jq --compact-output '.[]' | jq --compact-outpu [{"file_type":"archive","size":1049089,"filename":"artifacts.zip","file_format":"zip"},{"file_type":"metadata","size":157,"filename":"metadata.gz","file_format":"gzip"},{"file_type":"trace","size":3140,"filename":"job.log","file_format":null}] ``` -While the shell approach with `glab` works for smaller analysis, you should consider a script that -uses the API client libraries. This improves readability, storing data, flow control, testing, and reusability. - -You can also implement this algorithm with a Python script that uses the `python-gitlab` library: +:::TabTitle Python ```python #!/usr/bin/env python @@ -266,6 +290,8 @@ if __name__ == "__main__": print("DEBUG: ID {i}: {a}".format(i=job.id, a=job.attributes['artifacts'])) ``` +::EndTabs + The script outputs the project job artifacts in a JSON formatted list: ```json @@ -291,47 +317,28 @@ The script outputs the project job artifacts in a JSON formatted list: ] ``` -The full script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` with specific examples for automating storage management and cleanup is located is located in the [GitLab API with Python](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/) project. To ensure the script doesn't reach [API rate limits](../api/rest/index.md#rate-limits), the example code is not optimized for parallel API requests. +## Manage CI/CD pipeline storage -### Helper functions - -You may need to convert timestamp seconds into a duration format, or print raw bytes in a more -representative format. You can use the following helper functions to transform values for improved -readability: - -```shell -# Current Unix timestamp -date +%s - -# Convert `created_at` date time with timezone to Unix timestamp -date -d '2023-08-08T18:59:47.581Z' +%s -``` - -Example with Python that uses the `python-gitlab` API library: - -```python -def render_size_mb(v): - return "%.4f" % (v / 1024 / 1024) +Job artifacts consume most of the pipeline storage, and job logs can also generate several hundreds of kilobytes. +You should delete the unnecessary job artifacts first and then clean up job logs after analysis. -def render_age_time(v): - return str(datetime.timedelta(seconds = v)) +WARNING: +Deleting job log and artifacts is a destructive action that cannot be reverted. Use with caution. Deleting certain files, including report artifacts, job logs, and metadata files, affects GitLab features that use these files as data sources. -# Convert `created_at` date time with timezone to Unix timestamp -def calculate_age(created_at_datetime): - created_at_ts = datetime.datetime.strptime(created_at_datetime, '%Y-%m-%dT%H:%M:%S.%fZ') - now = datetime.datetime.now() - return (now - created_at_ts).total_seconds() -``` +### List job artifacts -## Managing storage in CI/CD pipelines +To analyze pipeline storage, you can use the [Job API endpoint](../api/jobs.md#list-project-jobs) to retrieve a list of +job artifacts. The endpoint returns the job artifacts `file_type` key in the `artifacts` attribute. +The `file_type` key indicates the artifact type: -WARNING: -Deleting job log and artifacts is a destructive action that cannot be reverted. Use with caution. Deleting certain files, including report artifacts, job logs, and metadata files, affects GitLab features that use these files as data sources. +- `archive` is used for the generated job artifacts as a zip file. +- `metadata` is used for additional metadata in a Gzip file. +- `trace` is used for the `job.log` as a raw file. -Job artifacts consume most of the pipeline storage, and job logs can also generate several hundreds of kilobytes. -You should delete the unnecessary job artifacts first and then clean up job logs after analysis. +Job artifacts provide a data structure that can be written as a cache file to +disk, which you can use to test the implementation. -### Analyze pipeline storage +Based on the example code for fetching all projects, you can extend the Python script to do more analysis. The following example shows a response from a query for job artifacts in a project: @@ -358,25 +365,19 @@ The following example shows a response from a query for job artifacts in a proje ] ``` -The [Job API endpoint](../api/jobs.md#list-project-jobs) returns the job artifacts `file_type` key in the `artifacts` attribute. The the job artifacts `file_type` key provides insights into the specific artifact type: - -- `archive` is used for the generated job artifacts as a zip file. -- `metadata` is used for additional metadata in a Gzip file. -- `trace` is used for the `job.log` as a raw file. - -These three types are relevant for storage counting, and should be collected for a later summary. Based on the example code for fetching all projects, you can extend the Python script to do more analysis. - -The Python code loops over all projects, and fetches a `project_obj` object variable that contains all attributes. Because there can be many pipelines and jobs, fetching the list of jobs can be expensive in one call. Therefore, this is done using [keyset pagination](https://python-gitlab.readthedocs.io/en/stable/api-usage.html#pagination). The remaining step is to fetch the `artifacts` attribute from the `job` object. - Based on how you implement the script, you could either: - Collect all job artifacts and print a summary table at the end of the script. - Print the information immediately. -Collecting the job artifacts provides a data structure that can be written as a cache file to -disk for example, which you can use when testing the implementation. +In the following example, job artifacts are collected in the `ci_job_artifacts` list. The script +loops over all projects, and fetches: -In the following example, the job artifacts are collected in the `ci_job_artifacts` list. +- The `project_obj` object variable that contains all attributes. +- The `artifacts` attribute from the `job` object. + +You can use [keyset pagination](https://python-gitlab.readthedocs.io/en/stable/api-usage.html#pagination) +to iterate over large lists of pipelines and jobs. ```python ci_job_artifacts = [] @@ -415,7 +416,8 @@ In the following example, the job artifacts are collected in the `ci_job_artifac print("No artifacts found.") ``` -At the end of the script, the job artifacts are printed as a Markdown formatted table. You can copy the table content into a new issue comment or description, or populate a Markdown file in a GitLab repository. +At the end of the script, job artifacts are printed as a Markdown formatted table. You can copy the table +content to an issue comment or description, or populate a Markdown file in a GitLab repository. ```shell $ python3 get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py @@ -430,22 +432,22 @@ $ python3 get_all_projects_top_level_namespace_storage_analysis_cleanup_example. | [gitlab-de/playground/artifact-gen-group/gen-job-artifacts-4](Gen Job Artifacts 4) | 4828297945 | job.log | trace | 0.0030 | ``` -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located in the [GitLab API with Python project](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/). To ensure the script doesn't hit [API rate limits](../api/rest/index.md#rate-limits), the example code is not optimized for parallel API requests. +### Delete job artifacts in bulk + +You can use a Python script to filter the types of job artifacts to delete in bulk. -### Delete job artifacts +Filter the API queries results to compare: -You can use a filter to select the types of job artifacts to delete in bulk. A typical request: +- The `created_at` value to calculate the artifact age. +- The `size` attribute to determine if artifacts meet the size threshold. + +A typical request: - Deletes job artifacts older than the specified number of days. - Deletes job artifacts that exceed a specified amount of storage. For example, 100 MB. -You can use a Python script to implement this type of filter. You can filter the API queries results, and compare -the `created_at` value to calculate the artifact age. - -You can also loop over all job artifacts and compare their `size` attribute to see whether they match -the size threshold. When a matching job has been found, it is marked for deletion. Because of the -analysis that happens when the script loops through job attributes, the job can be marked as deleted -only. When the collection loops remove the object locks, all marked as deleted jobs can actually be deleted. +In the following example, the script loops through job attributes and marks them for deletion. +When the collection loops remove the object locks, the script deletes the job artifacts marked for deletion. ```python for project in projects: @@ -489,18 +491,22 @@ only. When the collection loops remove the object locks, all marked as deleted j # Print collection summary (removed for readability) ``` -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located in the [GitLab API Python project](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/). - -#### Delete all job artifacts for a project +### Delete all job artifacts for a project If you do not need the project's [job artifacts](../ci/jobs/job_artifacts.md), you can -use the following command to delete them all. This action cannot be reverted. +use the following command to delete all job artifacts. This action cannot be reverted. + +Artifact deletion can take several minutes or hours, depending on the number of artifacts to delete. Subsequent +analysis queries against the API might return the artifacts as a false-positive result. +To avoid confusion with results, do not immediately run additional API requests. -Job artifact deletion happens asynchronously in GitLab and can take a while to complete in the background. Subsequent analysis queries against the API can still return the artifacts as a false-positive result. Artifact deletion can take minutes or hours, depending on the artifacts to delete. To avoid confusion with results, do not run immediate additional API requests. +The [artifacts for the most recent successful jobs](../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) are kept by default. -The [artifacts for the most recent successful jobs](../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) are also kept by default. +To delete all job artifacts for a project: -Example with curl: +::Tabs + +:::TabTitle curl ```shell export GL_PROJECT_ID=48349590 @@ -508,7 +514,7 @@ export GL_PROJECT_ID=48349590 curl --silent --header "Authorization: Bearer $GITLAB_TOKEN" --request DELETE "https://gitlab.com/api/v4/projects/$GL_PROJECT_ID/artifacts" ``` -Example with the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): +:::TabTitle GitLab CLI ```shell glab api --method GET projects/$GL_PROJECT_ID/jobs | jq --compact-output '.[]' | jq --compact-output '.id, .artifacts' @@ -516,17 +522,19 @@ glab api --method GET projects/$GL_PROJECT_ID/jobs | jq --compact-output '.[]' | glab api --method DELETE projects/$GL_PROJECT_ID/artifacts ``` -Example with the [`python-gitlab` library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#jobs): +:::TabTitle Python ```python project.artifacts.delete() ``` +::EndTabs + ### Delete job logs When you delete a job log you also [erase the entire job](../api/jobs.md#erase-a-job). -Example with the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): +Example with the GitLab CLI: ```shell glab api --method GET projects/$GL_PROJECT_ID/jobs | jq --compact-output '.[]' | jq --compact-output '.id' @@ -541,9 +549,9 @@ glab api --method POST projects/$GL_PROJECT_ID/jobs/4836226180/erase | jq --comp "success" ``` -In the `python-gitlab` API library, you must use [`job.erase()`](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#jobs) instead of `job.delete_artifacts()`. +In the `python-gitlab` API library, use [`job.erase()`](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#jobs) instead of `job.delete_artifacts()`. To avoid this API call from being blocked, set the script to sleep for a short amount of time between calls -that delete the job artifact. +that delete the job artifact: ```python for job in jobs_marked_delete_artifacts: @@ -555,20 +563,101 @@ that delete the job artifact. time.sleep(1) ``` -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located in the [GitLab API with Python project](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/). - Support for creating a retention policy for job logs is proposed in [issue 374717](https://gitlab.com/gitlab-org/gitlab/-/issues/374717). -### Inventory of job artifacts expiry settings +### Delete old pipelines + +Pipelines do not add to the overall storage consumption, but if required you can delete them with the following methods. + +Automatic deletion of old pipelines is proposed in [issue 338480](https://gitlab.com/gitlab-org/gitlab/-/issues/338480). + +Example with the GitLab CLI: + +```shell +export GL_PROJECT_ID=48349590 + +glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.id,.created_at' +960031926 +"2023-08-08T22:09:52.745Z" +959884072 +"2023-08-08T18:59:47.581Z" + +glab api --method DELETE projects/$GL_PROJECT_ID/pipelines/960031926 + +glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.id,.created_at' +959884072 +"2023-08-08T18:59:47.581Z" +``` + +The `created_at` key must be converted from a timestamp to Unix epoch time, +for example with `date -d '2023-08-08T18:59:47.581Z' +%s`. In the next step, the +age can be calculated with the difference between now, and the pipeline creation +date. If the age is larger than the threshold, the pipeline should be deleted. + +The following example uses a Bash script that expects `jq` and the GitLab CLI installed, and authorized, and the exported environment variable `GL_PROJECT_ID`. + +The full script `get_cicd_pipelines_compare_age_threshold_example.sh` is located in the [GitLab API with Linux Shell](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-linux-shell) project. + +```shell +#/bin/bash + +CREATED_AT_ARR=$(glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.created_at' | jq --raw-output @sh) + +for row in ${CREATED_AT_ARR[@]} +do + stripped=$(echo $row | xargs echo) + #echo $stripped #DEBUG + + CREATED_AT_TS=$(date -d "$stripped" +%s) + NOW=$(date +%s) + + AGE=$(($NOW-$CREATED_AT_TS)) + AGE_THRESHOLD=$((90*24*60*60)) # 90 days + + if [ $AGE -gt $AGE_THRESHOLD ]; + then + echo "Pipeline age $AGE older than threshold $AGE_THRESHOLD, should be deleted." + # TODO call glab to delete the pipeline. Needs an ID collected from the glab call above. + else + echo "Pipeline age $AGE not older than threshold $AGE_THRESHOLD. Ignore." + fi +done +``` + +You can use the [`python-gitlab` API library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#project-pipelines) and +the `created_at` attribute to implement a similar algorithm that compares the job artifact age: + +```python + # ... + + for pipeline in project.pipelines.list(iterator=True): + pipeline_obj = project.pipelines.get(pipeline.id) + print("DEBUG: {p}".format(p=json.dumps(pipeline_obj.attributes, indent=4))) + + created_at = datetime.datetime.strptime(pipeline.created_at, '%Y-%m-%dT%H:%M:%S.%fZ') + now = datetime.datetime.now() + age = (now - created_at).total_seconds() + + threshold_age = 90 * 24 * 60 * 60 + + if (float(age) > float(threshold_age)): + print("Deleting pipeline", pipeline.id) + pipeline_obj.delete() +``` + +### List expiry settings for job artifacts To manage artifact storage, you can update or configure when an artifact expires. The expiry setting for artifacts are configured in each job configuration in the `.gitlab-ci.yml`. -If you have multiple projects, and depending on how job definitions are organized in the CI/CD configuration, it may be difficult to locate the expiry setting. You can use a script to search the entire CI/CD configuration. This includes access to objects that are resolved after inheriting values, like `extends` or `!reference`. +If there are multiple projects, and based on how job definitions are organized in the CI/CD configuration, it might be difficult +to locate the expiry setting. You can use a script to search the entire CI/CD configuration. This includes access to objects that +are resolved after they inherit values, like `extends` or `!reference`. + The script retrieves merged CI/CD configuration files and searches for the artifacts key to: -- Identify the jobs that don't have an expiry setting. -- Return the expiry setting for jobs that have the artifact expiry configured. +- Identify jobs that do not have an expiry setting. +- Return expiry settings for jobs that have the artifact expiry configured. The following process describes how the script searches for the artifact expiry setting: @@ -626,7 +715,16 @@ The following process describes how the script searches for the artifact expiry print(f'| [{ details["project_name"] }]({details["project_web_url"]}) | { details["job_name"] } | { details["artifacts_expiry"] if details["artifacts_expiry"] is not None else "❌ N/A" } |') ``` -The script generates a Markdown summary table with project name and URL, job name, and the `artifacts:expire_in` setting, or `N/A` if not existing. It does not print job templates starting with a `.` character which are not instantiated as runtime job objects that would generate artifacts. +The script generates a Markdown summary table with: + +- Project name and URL. +- Job name. +- The `artifacts:expire_in` setting, or `N/A` if there is no setting. + +The script does not print job templates that: + +- Start with a `.` character. +- Are not instantiated as runtime job objects that generate artifacts. ```shell export GL_GROUP_ID=56595735 @@ -660,9 +758,9 @@ glab api --method GET projects/$GL_PROJECT_ID/search --field "scope=blobs" --fie For more information about the inventory approach, see [How GitLab can help mitigate deletion of open source container images on Docker Hub](https://about.gitlab.com/blog/2023/03/16/how-gitlab-can-help-mitigate-deletion-open-source-images-docker-hub/). -### Set the default expiry for job artifacts in projects +### Set default expiry for job artifacts -Based on the output of the `get_all_cicd_config_artifacts_expiry.py` script, you can define the [default artifact expiration](../ci/yaml/index.md#default) in your `.gitlab-ci.yml` configuration. +To set the default expiry for job artifacts in a project, specify the `expire_in` value in the `.gitlab-ci.yml` file: ```yaml default: @@ -670,93 +768,17 @@ default: expire_in: 1 week ``` -### Delete old pipelines - -Pipelines do not add to the overall storage consumption, but if you want to delete them you can use the following methods. - -Example using the [GitLab CLI](../editor_extensions/gitlab_cli/index.md): - -```shell -export GL_PROJECT_ID=48349590 - -glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.id,.created_at' -960031926 -"2023-08-08T22:09:52.745Z" -959884072 -"2023-08-08T18:59:47.581Z" - -glab api --method DELETE projects/$GL_PROJECT_ID/pipelines/960031926 - -glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.id,.created_at' -959884072 -"2023-08-08T18:59:47.581Z" -``` - -The `created_at` key must be converted from a timestamp to Unix epoch time, -for example with `date -d '2023-08-08T18:59:47.581Z' +%s`. In the next step, the -age can be calculated with the difference between now, and the pipeline creation -date. If the age is larger than the threshold, the pipeline should be deleted. - -The following example uses a Bash script that expects `jq` and the [GitLab CLI](../editor_extensions/gitlab_cli/index.md) installed, and authorized, and the exported environment variable `GL_PROJECT_ID`. - -The full script `get_cicd_pipelines_compare_age_threshold_example.sh` is located in the [GitLab API with Linux Shell](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-linux-shell) project. - -```shell -#/bin/bash - -CREATED_AT_ARR=$(glab api --method GET projects/$GL_PROJECT_ID/pipelines | jq --compact-output '.[]' | jq --compact-output '.created_at' | jq --raw-output @sh) - -for row in ${CREATED_AT_ARR[@]} -do - stripped=$(echo $row | xargs echo) - #echo $stripped #DEBUG - - CREATED_AT_TS=$(date -d "$stripped" +%s) - NOW=$(date +%s) - - AGE=$(($NOW-$CREATED_AT_TS)) - AGE_THRESHOLD=$((90*24*60*60)) # 90 days - - if [ $AGE -gt $AGE_THRESHOLD ]; - then - echo "Pipeline age $AGE older than threshold $AGE_THRESHOLD, should be deleted." - # TODO call glab to delete the pipeline. Needs an ID collected from the glab call above. - else - echo "Pipeline age $AGE not older than threshold $AGE_THRESHOLD. Ignore." - fi -done -``` - -You can use the [`python-gitlab` API library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/pipelines_and_jobs.html#project-pipelines) and -the `created_at` attribute to implement a similar algorithm that compares the job artifact age: +## Manage Container Registries storage -```python - # ... +Container registries are available [in a project](../api/container_registry.md#within-a-project) or [in a group](../api/container_registry.md#within-a-group). You can analyze both locations to implement a cleanup strategy. - for pipeline in project.pipelines.list(iterator=True): - pipeline_obj = project.pipelines.get(pipeline.id) - print("DEBUG: {p}".format(p=json.dumps(pipeline_obj.attributes, indent=4))) +### List container registries - created_at = datetime.datetime.strptime(pipeline.created_at, '%Y-%m-%dT%H:%M:%S.%fZ') - now = datetime.datetime.now() - age = (now - created_at).total_seconds() +To list Container Registries in a project: - threshold_age = 90 * 24 * 60 * 60 +::Tabs - if (float(age) > float(threshold_age)): - print("Deleting pipeline", pipeline.id) - pipeline_obj.delete() -``` - -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located in the [GitLab API with Python project](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/). - -Automatically deleting old pipelines in GitLab is tracked in [this feature proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/338480). - -## Manage storage for Container Registries - -Container registries are available [in a project](../api/container_registry.md#within-a-project) or [in a group](../api/container_registry.md#within-a-group). Both locations require analysis and cleanup strategies. - -The following example uses using `curl` and `jq` for a project: +:::TabTitle curl ```shell export GL_PROJECT_ID=48057080 @@ -771,7 +793,7 @@ curl --silent --header "Authorization: Bearer $GITLAB_TOKEN" "https://gitlab.com 3401613 ``` -The following example uses the [GitLab CLI](../editor_extensions/gitlab_cli/index.md) for a project: +:::TabTitle GitLab CLI ```shell export GL_PROJECT_ID=48057080 @@ -794,9 +816,9 @@ glab api --method GET projects/$GL_PROJECT_ID/registry/repositories/4435617/tags 3401613 ``` -A similar automation shell script is created in the [delete old pipelines](#delete-old-pipelines) section. +::EndTabs -The `python-gitlab` API library provides bulk deletion interfaces explained in the next section. +A similar automation shell script is created in the [delete old pipelines](#delete-old-pipelines) section. ### Delete container images in bulk @@ -810,7 +832,7 @@ you can configure: WARNING: On GitLab.com, due to the scale of the Container Registry, the number of tags deleted by this API is limited. If your Container Registry has a large number of tags to delete, only some of them are deleted. You might need -to call the API multiple times. To schedule tags for automatic deletion, use a [cleanup policy](#cleanup-policy-for-containers) instead. +to call the API multiple times. To schedule tags for automatic deletion, use a [cleanup policy](#create-a-cleanup-policy-for-containers) instead. The following example uses the [`python-gitlab` API library](https://python-gitlab.readthedocs.io/en/stable/gl_objects/repository_tags.html) to fetch a list of tags, and calls the `delete_in_bulk()` method with filter parameters. @@ -828,18 +850,17 @@ The following example uses the [`python-gitlab` API library](https://python-gitl repository.tags.delete_in_bulk(name_regex_delete="v.+", keep_n=2) ``` -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located -in the [GitLab API with Python](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/) project. +### Create a cleanup policy for containers -### Cleanup policy for containers +Use the project REST API endpoint to [create cleanup policies](packages/container_registry/reduce_container_registry_storage.md#use-the-cleanup-policy-api) for containers. After you set the cleanup policy, all container images that match your specifications are deleted automatically. You do not need additional API automation scripts. -Use the project REST API endpoint to [create cleanup policies](packages/container_registry/reduce_container_registry_storage.md#use-the-cleanup-policy-api). The following example uses the [GitLab CLI](../editor_extensions/gitlab_cli/index.md) to create a cleanup policy. - -To send the attributes as a body parameter, you must: +To send the attributes as a body parameter: - Use the `--input -` parameter to read from the standard input. - Set the `Content-Type` header. +The following example uses the GitLab CLI to create a cleanup policy: + ```shell export GL_PROJECT_ID=48057080 @@ -859,19 +880,17 @@ echo '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":tr ``` -After you set up the cleanup policy, all container images that match your specifications are deleted automatically. You do not need additional API automation scripts. - ### Optimize container images You can optimize container images to reduce the image size and overall storage consumption in the container registry. Learn more in the [pipeline efficiency documentation](../ci/pipelines/pipeline_efficiency.md#optimize-docker-images). -## Manage storage for Package Registry +## Manage Package Registry storage Package registries are available [in a project](../api/packages.md#within-a-project) or [in a group](../api/packages.md#within-a-group). ### List packages and files -The following example shows fetching packages from a defined project ID using the [GitLab CLI](../editor_extensions/gitlab_cli/index.md). The result set is an array of dictionary items that can be filtered with the `jq` command chain. +The following example shows fetching packages from a defined project ID using the GitLab CLI. The result set is an array of dictionary items that can be filtered with the `jq` command chain. ```shell # https://gitlab.com/gitlab-de/playground/container-package-gen-group/generic-package-generator @@ -923,7 +942,7 @@ and loops over its package files to print the `file_name` and `size` attributes. [Deleting a file in a package](../api/packages.md#delete-a-package-file) can corrupt the package. You should delete the package when performing automated cleanup maintenance. -To delete a package, use the [GitLab CLI](../editor_extensions/gitlab_cli/index.md) to change the `--method` +To delete a package, use the GitLab CLI to change the `--method` parameter to `DELETE`: ```shell @@ -981,18 +1000,39 @@ Package size: 20.0033 Package size 20.0033 > threshold 10.0000, deleting package. ``` -The full example of the script `get_all_projects_top_level_namespace_storage_analysis_cleanup_example.py` is located in the [GitLab API with Python](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-api-python/) project. - ### Dependency Proxy Review the [cleanup policy](packages/dependency_proxy/reduce_dependency_proxy_storage.md#cleanup-policies) and how to [purge the cache using the API](packages/dependency_proxy/reduce_dependency_proxy_storage.md#use-the-api-to-clear-the-cache) -## Community resources +## Improve output readability -These resources are not officially supported. Ensure to test scripts and tutorials before running destructive cleanup commands that may not be reverted. +You might need to convert timestamp seconds into a duration format, or print raw bytes in a more +representative format. You can use the following helper functions to transform values for improved +readability: -- Forum topic: [Storage management automation resources](https://forum.gitlab.com/t/storage-management-automation-resources/) -- Script: [GitLab Storage Analyzer](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-storage-analyzer), unofficial project by the [GitLab Developer Evangelism team](https://gitlab.com/gitlab-de/). You find similar code examples in this documentation how-to here. +```shell +# Current Unix timestamp +date +%s + +# Convert `created_at` date time with timezone to Unix timestamp +date -d '2023-08-08T18:59:47.581Z' +%s +``` + +Example with Python that uses the `python-gitlab` API library: + +```python +def render_size_mb(v): + return "%.4f" % (v / 1024 / 1024) + +def render_age_time(v): + return str(datetime.timedelta(seconds = v)) + +# Convert `created_at` date time with timezone to Unix timestamp +def calculate_age(created_at_datetime): + created_at_ts = datetime.datetime.strptime(created_at_datetime, '%Y-%m-%dT%H:%M:%S.%fZ') + now = datetime.datetime.now() + return (now - created_at_ts).total_seconds() +``` ## Testing for storage management automation @@ -1143,3 +1183,10 @@ Use the following projects to test storage usage with [cost factors for forks](u - Fork [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) into a new namespace or group (includes LFS, Git repository). - Fork [`gitlab-com/www-gitlab-com`](https://gitlab.com/gitlab-com/www-gitlab-comgitlab-com/www-gitlab-com) into a new namespace or group. + +## Community resources + +The following resources are not officially supported. Ensure to test scripts and tutorials before running destructive cleanup commands that may not be reverted. + +- Forum topic: [Storage management automation resources](https://forum.gitlab.com/t/storage-management-automation-resources/) +- Script: [GitLab Storage Analyzer](https://gitlab.com/gitlab-de/use-cases/gitlab-api/gitlab-storage-analyzer), unofficial project by the [GitLab Developer Evangelism team](https://gitlab.com/gitlab-de/). You find similar code examples in this documentation how-to here. diff --git a/doc/user/tasks.md b/doc/user/tasks.md index f4fec01d42b..347aedd6e74 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -443,3 +443,50 @@ The description and threads are on the left, and attributes, such as labels or assignees, on the right. ![Task two column view](img/task_two_column_view_v16_2.png) + +## Linked items in Tasks + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/416558) in GitLab 16.5 [with a flag](../administration/feature_flags.md) named `linked_work_items`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `linked_work_items`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +Linked items are a bi-directional relationship and appear in a block below +the emoji reactions section. You can link an objective, key result, or a task in the same project with each other. + +The relationship only shows up in the UI if the user can see both items. + +### Add a linked item + +Prerequisite: + +- You must have at least the Guest role for the project. + +To link an item to a task: + +1. In the **Linked items** section of a task, + select the **Add** button. +1. Select the relationship between the two items. Either: + - **relates to** + - **blocks** + - **is blocked by** +1. Enter the search text of the item. +1. When you have added all the items to be linked, select **Add** below the search box. + +When you have finished adding all linked items, you can see +them categorized so their relationships can be better understood visually. + +![Linked items block](img/linked_items_list_v16_5.png) + +### Remove a linked item + +Prerequisite: + +- You must have at least the Guest role for the project. + +In the **Linked items** section of a task, +next to each item, select the vertical ellipsis (**{ellipsis_v}**) and then select **Remove**. + +Due to the bi-directional relationship, the relationship no longer appears in either item. diff --git a/doc/user/todos.md b/doc/user/todos.md index eedd5d8a510..8e207a786c3 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -66,11 +66,12 @@ To-do items aren't affected by [GitLab notification email settings](profile/noti > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28355) in GitLab 13.8 [with a flag](../administration/feature_flags.md) named `multiple_todos`. Disabled by default. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82470) in GitLab 14.9: only mentions create multiple to-do items. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/28355) in GitLab 16.2. FLAG: On self-managed GitLab, by default this feature is not available. To make it available per user, an administrator can [enable the feature flag](../administration/feature_flags.md) named `multiple_todos`. -On GitLab.com, this feature is not available. +On GitLab.com, this feature is available. The feature is not ready for production use. When you enable this feature: diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 8c6840fae92..305a46e1f15 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -16,6 +16,15 @@ Statistics include: - Storage usage that exceeds the storage quota. - Available purchased storage. +Storage and network usage are calculated with the binary measurement system (1024 unit multiples). +Storage usage is displayed in kibibytes (KiB), mebibytes (MiB), +or gibibytes (GiB). 1 KiB is 2^10 bytes (1024 bytes), +1 MiB is 2^20 bytes (1024 kibibytes), 1 GiB is 2^30 bytes (1024 mebibytes). + +NOTE: +Storage usage labels are being transitioned from `KB` to `KiB`, `MB` to `MiB`, and `GB` to `GiB`. During this transition, +you might see references to `KB`, `MB`, and `GB` in the UI and documentation. + ## View storage usage Prerequisites: @@ -123,7 +132,7 @@ Depending on your role, to manage your transfer usage you can [reduce Container ## Project storage limit -Projects on GitLab SaaS have a 10 GB storage limit on their Git repository and LFS storage. +Projects on GitLab SaaS have a 10 GiB storage limit on their Git repository and LFS storage. After namespace-level storage limits are applied, the project limit is removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both. When a project's repository and LFS reaches the quota, the project is set to a read-only state. @@ -153,28 +162,28 @@ The following example describes an excess storage scenario for a namespace: | Repository | Storage used | Excess storage | Quota | Status | |------------|--------------|----------------|--------|----------------------| -| Red | 10 GB | 0 GB | 10 GB | Read-only **{lock}** | -| Blue | 8 GB | 0 GB | 10 GB | Not read-only | -| Green | 10 GB | 0 GB | 10 GB | Read-only **{lock}** | -| Yellow | 2 GB | 0 GB | 10 GB | Not read-only | -| **Totals** | **30 GB** | **0 GB** | - | - | +| Red | 10 GiB | 0 GiB | 10 GiB | Read-only **{lock}** | +| Blue | 8 GiB | 0 GiB | 10 GiB | Not read-only | +| Green | 10 GiB | 0 GiB | 10 GiB | Read-only **{lock}** | +| Yellow | 2 GiB | 0 GiB | 10 GiB | Not read-only | +| **Totals** | **30 GiB** | **0 GiB** | - | - | The Red and Green projects are read-only because their repositories and LFS have reached the quota. In this example, no additional storage has yet been purchased. -To remove the read-only state from the Red and Green projects, 50 GB additional storage is purchased. +To remove the read-only state from the Red and Green projects, 50 GiB additional storage is purchased. -Assuming the Green and Red projects' repositories and LFS grow past the 10 GB quota, the purchased storage -available decreases. All projects no longer have the read-only status because 40 GB purchased storage is available: -50 GB (purchased storage) - 10 GB (total excess storage used). +Assuming the Green and Red projects' repositories and LFS grow past the 10 GiB quota, the purchased storage +available decreases. All projects no longer have the read-only status because 40 GiB purchased storage is available: +50 GiB (purchased storage) - 10 GiB (total excess storage used). | Repository | Storage used | Excess storage | Quota | Status | |------------|--------------|----------------|---------|-------------------| -| Red | 15 GB | 5 GB | 10 GB | Not read-only | -| Blue | 14 GB | 4 GB | 10 GB | Not read-only | -| Green | 11 GB | 1 GB | 10 GB | Not read-only | -| Yellow | 5 GB | 0 GB | 10 GB | Not read-only | -| **Totals** | **45 GB** | **10 GB** | - | - | +| Red | 15 GiB | 5 GiB | 10 GiB | Not read-only | +| Blue | 14 GiB | 4 GiB | 10 GiB | Not read-only | +| Green | 11 GiB | 1 GiB | 10 GiB | Not read-only | +| Yellow | 5 GiB | 0 GiB | 10 GiB | Not read-only | +| **Totals** | **45 GiB** | **10 GiB** | - | - | ## Namespace storage limit @@ -212,7 +221,7 @@ To prevent exceeding the namespace storage limit, you can: - [GitLab for Open Source](https://about.gitlab.com/solutions/open-source/join/) - [GitLab for Startups](https://about.gitlab.com/solutions/startups/) - Consider using a [self-managed instance](../subscriptions/self_managed/index.md) of GitLab, which does not have these limits on the Free tier. -- [Purchase additional storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) units at $60/year for 10 GB of storage. +- [Purchase additional storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) units at $60 per year for 10 GiB of storage. - [Start a trial](https://about.gitlab.com/free-trial/) or [upgrade to GitLab Premium or Ultimate](https://about.gitlab.com/pricing/), which include higher limits and features to enable growing teams to ship faster without sacrificing on quality. - [Talk to an expert](https://page.gitlab.com/usage_limits_help.html) for more information about your options. |