diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-05-17 19:05:49 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-05-17 19:05:49 +0300 |
commit | 43a25d93ebdabea52f99b05e15b06250cd8f07d7 (patch) | |
tree | dceebdc68925362117480a5d672bcff122fb625b /doc/user | |
parent | 20c84b99005abd1c82101dfeff264ac50d2df211 (diff) |
Add latest changes from gitlab-org/gitlab@16-0-stable-eev16.0.0-rc42
Diffstat (limited to 'doc/user')
477 files changed, 11160 insertions, 7178 deletions
diff --git a/doc/user/admin_area/analytics/dev_ops_reports.md b/doc/user/admin_area/analytics/dev_ops_reports.md index 2d19c0a0058..31cc9825452 100644 --- a/doc/user/admin_area/analytics/dev_ops_reports.md +++ b/doc/user/admin_area/analytics/dev_ops_reports.md @@ -39,7 +39,7 @@ feature is available. ## DevOps Adoption **(ULTIMATE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../../policy/alpha-beta-support.md#beta). > - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. > - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. > - Fuzz Testing metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/330398) in GitLab 14.2. diff --git a/doc/user/admin_area/analytics/index.md b/doc/user/admin_area/analytics/index.md index 4304e612e4a..2ac8941b286 100644 --- a/doc/user/admin_area/analytics/index.md +++ b/doc/user/admin_area/analytics/index.md @@ -14,7 +14,7 @@ Instance-level analytics provide insights into the feature and data usage of you Prerequisite: -- You must have administrator access for your instance. +- You must have administrator access to the instance. To view instance-level analytics: diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md index a1fae7e8712..a5311b083c3 100644 --- a/doc/user/admin_area/appearance.md +++ b/doc/user/admin_area/appearance.md @@ -2,7 +2,6 @@ stage: none group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/customization/branded_login_page.html' --- # GitLab Appearance **(FREE SELF)** @@ -71,6 +70,27 @@ to review the saved appearance settings: NOTE: You can add also add a [customized help message](settings/help_page.md) below the sign in message or add [a Sign in text message](settings/sign_in_restrictions.md#sign-in-information). +## Progressive Web App + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375708) in GitLab 15.9. + +GitLab can be installed as a [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) (PWA). +Use the Progressive Web App settings to customize its appearance, including its name, +description, and icon. + +### Configure the PWA settings + +To configure the PWA settings: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Appearance**. +1. Scroll to the **Progressive Web App (PWA)** section. +1. Complete the fields. + - **Icon**: If you use the standard GitLab icon, it is available in sizes 192x192 pixels, + 512x512 pixels, also as a maskable icon. If you use a custom icon, it must be in either size + 192x192 pixels, or 512x512 pixels. +1. Select **Update appearance settings**. + ## New project pages You can add a new project guidelines message to the **New project page** in GitLab. diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md index 847f687d051..9d360539595 100644 --- a/doc/user/admin_area/custom_project_templates.md +++ b/doc/user/admin_area/custom_project_templates.md @@ -1,8 +1,7 @@ --- -stage: Manage -group: Import +stage: Create +group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference --- # Custom instance-level project templates **(PREMIUM SELF)** @@ -18,7 +17,7 @@ is created, based on the user's access permissions: - Public projects can be selected by any authenticated user as a template for a new project, if all enabled [project features](../project/settings/index.md#configure-project-visibility-features-and-permissions) - except for **GitLab Pages** and **Security & Compliance** are set to **Everyone With Access**. + except for **GitLab Pages** and **Security and Compliance** are set to **Everyone With Access**. The same applies to internal projects. - Private projects can be selected only by users who are members of the projects. @@ -41,6 +40,24 @@ To select the group to use as the source for the project templates: Projects in subgroups of the template group are **not** included in the template list. +## What is copied from the templates + +The entire custom instance-level project templates repository is copied, including: + +- Branches +- Commits +- Tags + +If the user: + +- Has the Owner role on the custom instance-level project templates project or is a GitLab administrator, all project settings are copied over to the new + project. +- Doesn't have the Owner role or is not a GitLab administrator, project [deploy keys](../project/deploy_keys/index.md#view-deploy-keys) and project + [webhooks](../project/integrations/webhooks.md) aren't copied over because they contain sensitive data. + +To learn more about what is migrated, see +[Items that are exported](../project/settings/import_export.md#items-that-are-exported). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/external_users.md b/doc/user/admin_area/external_users.md index 8b968a3da01..5127630d65e 100644 --- a/doc/user/admin_area/external_users.md +++ b/doc/user/admin_area/external_users.md @@ -48,6 +48,7 @@ Additionally, users can be set as external users using: - [SAML groups](../../integration/saml.md#external-groups). - [LDAP groups](../../administration/auth/ldap/ldap_synchronization.md#external-groups). +- the [External providers list](../../integration/omniauth.md#create-an-external-providers-list). ## Set a new user to external diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 0375232334f..71c2468c97f 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -86,6 +86,16 @@ project, the following information is listed: Projects can be edited or deleted. +To edit a project's name or description: + +1. In the Projects overview, next to the project you want to edit, select **Edit**. +1. Edit the **Project name** or **Project description**. +1. Select **Save Changes**. + +To delete a project: + +1. In the Projects overview, next to the project you want to delete, select **Delete**. + The list of projects can be sorted by: - Updated date @@ -448,7 +458,7 @@ For multi-node systems we recommend ingesting the logs into services like Elasti | Log file | Contents | |:------------------------|:---------| -| `application.log` | GitLab user activity | +| `application_json.log` | GitLab user activity | | `git_json.log` | Failed GitLab interaction with Git repositories | | `production.log` | Requests received from Puma, and the actions taken to serve those requests | | `sidekiq.log` | Background jobs | diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index 8e1ca979707..16721d144e5 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -10,6 +10,7 @@ type: reference To manage labels for the GitLab instance, select **Labels** (**{labels}**) from the Admin Area sidebar. For more details on how to manage labels, see [Labels](../project/labels.md). Labels created in the Admin Area are automatically added to new projects. +They are not available to new groups. Updating or adding labels in the Admin Area does not modify labels in existing projects. ![Default label set](img/admin_labels_v14_7.png) diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 5296a918f56..823f876539f 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -78,3 +78,5 @@ You may have connectivity issues due to the following reasons: - If the curl command returns a failure, either: - [Configure a proxy](https://docs.gitlab.com/omnibus/settings/environment-variables.html) in `gitlab.rb` to point to your server. - Contact your network administrator to make changes to the proxy. + - If an SSL inspection appliance is used, you must add the appliance's root CA certificate to `/etc/gitlab/trusted-certs` on the server, then run `gitlab-ctl reconfigure`. +
\ No newline at end of file diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md index 29e43476819..01d2c31dd10 100644 --- a/doc/user/admin_area/license_file.md +++ b/doc/user/admin_area/license_file.md @@ -28,9 +28,19 @@ NOTE: In GitLab 14.7.x to 14.9.x, you can add the license file with the UI. In GitLab 14.1.x to 14.7, if you have already activated your subscription with an activation code, you cannot access **Add License** from the Admin Area. You must access **Add License** directly from the URL, `<YourGitLabURL>/admin/license/new`. -## Add your license file during installation +## Activate subscription during installation -You can import a license file when you install GitLab. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114572) in GitLab 16.0. + +To activate your subscription during installation, set the `GITLAB_ACTIVATION_CODE` environment variable with the activation code: + +```shell +export GITLAB_ACTIVATION_CODE=your_activation_code +``` + +## Add license file during installation + +If you have a license, you can also import it when you install GitLab. - **For installations from source** - Place the `Gitlab.gitlab-license` file in the `config/` directory. @@ -183,6 +193,17 @@ License.current.license_id # License data in Base64-encoded ASCII format License.current.data + +# Confirm the current billable seat count excluding guest users. This is useful for customers who use an Ultimate subscription tier where Guest seats are not counted. +User.active.without_bots.excluding_guests.count + +``` + +#### Interaction with licenses that start in the future + +```ruby +# Future license data follows the same format as current license data it just uses a different modifier for the License prefix +License.future_dated ``` #### Check if a project feature is available on the instance diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index a273798c8eb..b0e24559e47 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -5,7 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Moderate users **(FREE SELF)** +# Moderate users (administration) **(FREE SELF)** + +This is the administration documentation. For information about moderating users at the group level, see the [group-level documentation](../group/moderate_users.md). GitLab administrators can moderate user access by approving, blocking, banning, or deactivating users. @@ -161,8 +163,8 @@ A user can be deactivated from the Admin Area. To do this: For the deactivation option to be visible to an administrator, the user: -- Must be currently active. -- Must not be [dormant](#automatically-deactivate-dormant-users). +- Must have a state of active. +- Must be [dormant](#automatically-deactivate-dormant-users). NOTE: Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). diff --git a/doc/user/admin_area/monitoring/background_migrations.md b/doc/user/admin_area/monitoring/background_migrations.md deleted file mode 100644 index b4a6f7f66fb..00000000000 --- a/doc/user/admin_area/monitoring/background_migrations.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../../../update/background_migrations.md' -remove_date: '2023-03-11' ---- - -This document was moved to [another location](../../../update/background_migrations.md). - -<!-- This redirect file can be deleted after <2023-03-11>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 668d34af024..f3b09c61532 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -92,8 +92,6 @@ Example response: On failure, the endpoint returns a `503` HTTP status code. -This check does hit the database and Redis if authenticated via `token`. - This check is being exempt from Rack Attack. ## Liveness diff --git a/doc/user/admin_area/reporting/git_abuse_rate_limit.md b/doc/user/admin_area/reporting/git_abuse_rate_limit.md index 66d1173058e..83b28404714 100644 --- a/doc/user/admin_area/reporting/git_abuse_rate_limit.md +++ b/doc/user/admin_area/reporting/git_abuse_rate_limit.md @@ -4,20 +4,16 @@ group: Anti-Abuse info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Git abuse rate limit (administration) **(ULTIMATE)** +# Git abuse rate limit (administration) **(ULTIMATE SELF)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/394996) in GitLab 15.10. Feature flag `git_abuse_rate_limit_feature_flag` removed. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. On GitLab.com, this feature is available. +This is the administration documentation. For information about Git abuse rate limiting at the group level, see the [group-level documentation](../../group/reporting/git_abuse_rate_limit.md). -Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download or clone more than a specified number of repositories in any project in the instance within a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. +Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download, clone, or fork more than a specified number of repositories in any project in the instance in a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). -If the `git_abuse_rate_limit_feature_flag` feature flag is enabled, all application administrators receive an email when a user is about to be banned. - -If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, administrators are still notified. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. - -If automatic banning is enabled, administrators receive an email when a user is about to be banned, and the user is automatically banned from the GitLab instance. +Git abuse rate limiting does not apply to instance administrators, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). ## Configure Git abuse rate limiting @@ -28,9 +24,16 @@ If automatic banning is enabled, administrators receive an email when a user is 1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled. 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled. 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned. + 1. Add up to `100` users to the **Send notifications to** field. You must select at least one user. All application administrators are selected by default. 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. 1. Select **Save changes**. +## Automatic ban notifications + +If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent to the users listed under **Send notifications to**. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. + +If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the GitLab instance. + ## Unban a user 1. On the top bar, select **Main menu > Admin**. diff --git a/doc/user/admin_area/reporting/spamcheck.md b/doc/user/admin_area/reporting/spamcheck.md index 5c305eff4fa..16c144d2469 100644 --- a/doc/user/admin_area/reporting/spamcheck.md +++ b/doc/user/admin_area/reporting/spamcheck.md @@ -21,15 +21,15 @@ Spamcheck is only available for package-based installations: 1. Edit `/etc/gitlab/gitlab.rb` and enable Spamcheck: - ```ruby - spamcheck['enable'] = true - ``` + ```ruby + spamcheck['enable'] = true + ``` 1. Reconfigure GitLab: - ```shell - sudo gitlab-ctl reconfigure - ``` + ```shell + sudo gitlab-ctl reconfigure + ``` 1. Verify that the new services `spamcheck` and `spam-classifier` are up and running: diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md index b8531fded18..314e0c77f36 100644 --- a/doc/user/admin_area/review_abuse_reports.md +++ b/doc/user/admin_area/review_abuse_reports.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Authentication and Authorization +stage: Anti-Abuse +group: Anti-Abuse info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference, howto --- diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 35a4c0aeea7..5c730375f98 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -36,7 +36,9 @@ can create in their personal namespace: ## Max attachment size -The maximum file size for attachments in GitLab comments and replies is 10 MB. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/20061) from 10 MB to 100 MB in GitLab 15.7. + +The maximum file size for attachments in GitLab comments and replies is 100 MB. To change the maximum attachment size: 1. On the top bar, select **Main menu > Admin**. @@ -174,7 +176,32 @@ wiki, packages, or snippets. The repository size limit applies to both private a For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). -## Customize session duration for Git Operations when 2FA is enabled **(PREMIUM SELF)** +## Session duration + +### Customize the default session duration + +You can change how long users can remain signed in without activity. + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Account and limit**. The set duration is in **Session duration (minutes)**. + +If [Remember me](#turn-remember-me-on-or-off) is enabled, users' sessions can remain active for an indefinite period of time. + +For details, see [cookies used for sign-in](../../profile/index.md#cookies-used-for-sign-in). + +### Turn **Remember me** on or off + +> Ability to turn the **Remember me** setting on and off [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/369133) in GitLab 16.0. + +Users can select the **Remember me** checkbox on sign-in, and their session will remain active for an indefinite period of time when accessed from that specific browser. You can turn off this setting if you need sessions to expire for security or compliance purposes. Turning off this setting will ensure users' sessions expire after the number of minutes of inactivity set when you [customize your session duration](#customize-the-default-session-duration). + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Account and limit**. +1. Select or clear the **Remember me** checkbox to turn this setting on or off. + +### Customize session duration for Git Operations when 2FA is enabled **(PREMIUM SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296669) in GitLab 13.9. > - It's deployed behind a feature flag, disabled by default. @@ -305,6 +332,17 @@ By default, newly created users have a public profile. GitLab administrators can 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Select the **Make new users' profiles private by default** checkbox. +## Prevent users from deleting their accounts **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26053) in GitLab 16.0 [with a flag](../../../administration/feature_flags.md) named `deleting_account_disabled_for_users`. Disabled by default. + +By default, users can delete their own accounts. GitLab administrators can prevent +users from deleting their own accounts: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. +1. Clear the **Allows users to delete their own accounts** checkbox. + ## Troubleshooting ### 413 Request Entity Too Large diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index aa171fe4536..27af64cd0e8 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -54,7 +54,7 @@ To enable a project runner for more than one project: 1. On the top bar, select **Main menu > Admin**. 1. From the left sidebar, select **CI/CD > Runners**. 1. Select the runner you want to edit. -1. In the upper right, select **Edit** (**{pencil}**). +1. In the upper-right corner, select **Edit** (**{pencil}**). 1. Under **Restrict projects for this runner**, search for a project. 1. To the left of the project, select **Enable**. 1. Repeat this process for each additional project. @@ -148,7 +148,7 @@ are locked against deletion and kept regardless of the expiry time. When disabled, the latest artifacts for any **new** successful or fixed pipelines are allowed to expire. -This setting takes precedence over the [project level setting](../../../ci/pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +This setting takes precedence over the [project level setting](../../../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). If disabled at the instance level, you cannot enable this per-project. To disable the setting: @@ -194,6 +194,16 @@ To set all new [CI/CD variables](../../../ci/variables/index.md) as 1. On the left sidebar, select **Settings > CI/CD**. 1. Select **Protect CI/CD variables by default**. +## Maximum includes + +The maximum number of [includes](../../../ci/yaml/includes.md) per pipeline can be set at the instance level. +The default is `150`. + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Change the value of **Maximum includes**. +1. Select **Save changes** for the changes to take effect. + ## Default CI/CD configuration file > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18073) in GitLab 12.5. @@ -227,6 +237,7 @@ from the Admin Area: - **Maximum number of DAG dependencies that a job can have** - **Maximum number of runners registered per group** - **Maximum number of runners registered per project** + - **Maximum number of downstream pipelines in a pipeline's hierarchy tree** ## Enable or disable the pipeline suggestion banner @@ -244,12 +255,13 @@ To enable or disable the banner: ## Required pipeline configuration **(ULTIMATE SELF)** -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9. -NOTE: -An alternative [compliance solution](../../group/compliance_frameworks.md#compliance-pipelines) -is available. We recommend this alternative solution because it provides greater flexibility, -allowing required pipelines to be assigned to specific compliance framework labels. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9 +and is planned for removal in 17.0. Use [compliance pipelines](../../group/compliance_frameworks.md#compliance-pipelines) +instead. This change is a breaking change. You can set a [CI/CD template](../../../ci/examples/index.md#cicd-templates) as a required pipeline configuration for all projects on a GitLab instance. You can @@ -267,7 +279,7 @@ use a template from: The project CI/CD configuration merges into the required pipeline configuration when a pipeline runs. The merged configuration is the same as if the required pipeline configuration added the project configuration with the [`include` keyword](../../../ci/yaml/index.md#include). -To view a project's full merged configuration, [View the merged YAML](../../../ci/pipeline_editor/index.md#view-expanded-configuration) +To view a project's full merged configuration, [View full configuration](../../../ci/pipeline_editor/index.md#view-full-configuration) in the pipeline editor. To select a CI/CD template for the required pipeline configuration: @@ -344,9 +356,9 @@ To restrict all users in an instance from registering runners: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **Runner registration**. -1. Clear the checkbox if you don't want to display runner registration - information in the UI for group or project members. +1. Expand **Runners**. +1. In the **Runner registration** section, clear the **Members of the project can register runners** and + **Members of the group can register runners** checkboxes to remove runner registration from the UI. 1. Select **Save changes**. NOTE: @@ -370,6 +382,20 @@ To restrict runner registration by members in a specific group: 1. Clear the **New group runners can be registered** checkbox if you want to disable runner registration by all members in the group. If the setting is read-only, you must enable runner registration for the [instance](#restrict-runner-registration-by-all-users-in-an-instance). 1. Select **Save changes**. +## Disable runner version management + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114041) in GitLab 15.10. + +By default, GitLab instances periodically fetch official runner version data from GitLab.com to [determine whether the runners need upgrades](../../../ci/runners/configure_runners.md#determine-which-runners-need-to-be-upgraded). + +To disable your instance fetching this data: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Runner version management** section, clear the **Fetch GitLab Runner release version data from GitLab.com** checkbox. +1. Select **Save changes**. + ## Troubleshooting ### 413 Request Entity Too Large diff --git a/doc/user/admin_area/settings/deprecated_api_rate_limits.md b/doc/user/admin_area/settings/deprecated_api_rate_limits.md index 8bf0ffd21a5..13f8bc008e3 100644 --- a/doc/user/admin_area/settings/deprecated_api_rate_limits.md +++ b/doc/user/admin_area/settings/deprecated_api_rate_limits.md @@ -28,9 +28,9 @@ the general user and IP rate limits for requests to deprecated endpoints. You ca and IP rate limits already in place, and increase or decrease the rate limits for deprecated API endpoints. No other new features are provided by this override. -Prerequisites: +Prerequisite: -- You must have administrator access for your instance. +- You must have administrator access to the instance. To override the general user and IP rate limits for requests to deprecated API endpoints: diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 484f51d8739..90852463e9d 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -86,12 +86,13 @@ To disable these notifications: ### Custom additional text in deactivation emails **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. +> - [Enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111882) in GitLab 15.9. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an -administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`deactivation_email_additional_text`. On GitLab.com, this feature is unavailable. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an +administrator to [disable the feature flag](../../../administration/feature_flags.md) named +`deactivation_email_additional_text`. You can add additional text at the bottom of the email that GitLab sends to users when their account is deactivated. This email text is separate from the [custom additional text](#custom-additional-text) diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index 94d9ec73640..072873ba7f6 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -13,8 +13,6 @@ controlled by an external service that permits access based on project classification and user access. GitLab provides a way to check project authorization with your own defined service. -## Overview - After the external service is configured and enabled, when a project is accessed, a request is made to the external service with the user information and project classification label assigned to the project. When the service @@ -39,13 +37,10 @@ the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/logs When using TLS Authentication with a self signed certificate, the CA certificate needs to be trusted by the OpenSSL installation. When using GitLab installed using Omnibus, learn to install a custom CA in the -[Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). +[Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/ssl/index.html). Alternatively, learn where to install custom certificates by using `openssl version -d`. -When external authorization is enabled, [deploy tokens](../../project/deploy_tokens/index.md) - and [deploy keys](../../project/deploy_keys/index.md) can't be used for Git operations. - ## Configuration The external authorization service can be enabled by an administrator: @@ -56,6 +51,30 @@ The external authorization service can be enabled by an administrator: 1. Complete the fields. 1. Select **Save changes**. +### Allow external authorization with deploy tokens and deploy keys + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386656) in GitLab 15.9. +> - Deploy tokens no longer being able to access container or package registries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387721) in GitLab 16.0. + +You can set your instance to allow external authorization for Git operations with +[deploy tokens](../../project/deploy_tokens/index.md) or [deploy keys](../../project/deploy_keys/index.md). + +Prerequisites: + +- You must be using classification labels without a service URL for external authorization. + +To allow authorization with deploy tokens and keys: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand **External authorization**, and: + - Leave the service URL field empty. + - Select **Allow deploy tokens and deploy keys to be used with external authorization**. +1. Select **Save changes**. + +WARNING: +If you enable external authorization, deploy tokens cannot access container or package registries. If you use deploy tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use tokens with container or package registries. + ## How it works When GitLab requests access, it sends a JSON POST request to the external @@ -106,7 +125,7 @@ You can use your own classification label in the project's label" box. When no classification label is specified on a project, the default label defined in the [global settings](#configuration) is used. -The label is shown on all project pages in the upper right corner. +On all project pages, in the upper-right corner, the label appears. ![classification label on project page](img/classification_label_on_project_page_v14_8.png) diff --git a/doc/user/admin_area/settings/files_api_rate_limits.md b/doc/user/admin_area/settings/files_api_rate_limits.md index ef9a3674c49..8677e3d86bf 100644 --- a/doc/user/admin_area/settings/files_api_rate_limits.md +++ b/doc/user/admin_area/settings/files_api_rate_limits.md @@ -26,7 +26,7 @@ for the Files API. No other new features are provided by this override. Prerequisite: -- You must have administrator access for your instance. +- You must have administrator access to the instance. To override the general user and IP rate limits for requests to the Repository files API: diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md index 07d3ae83d74..5d9fc23aaff 100644 --- a/doc/user/admin_area/settings/help_page.md +++ b/doc/user/admin_area/settings/help_page.md @@ -71,14 +71,7 @@ You can specify a custom URL to which users are directed when they: > - [Feature flag `help_page_documentation_redirect`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) removed in GitLab 14.4. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) in GitLab 14.4. -The `/help` URL of a GitLab instance displays a basic version of the documentation sourced from the -[`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. `/help` links -are often used for contextual help. - -You can redirect these `/help` links to either: - -- The more navigable and searchable version published at [`docs.gitlab.com`](https://docs.gitlab.com). -- A destination that meets [necessary requirements](#destination-requirements). +You can redirect all `/help` links to a destination that meets the [necessary requirements](#destination-requirements). 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Preferences**. @@ -86,17 +79,19 @@ You can redirect these `/help` links to either: 1. In the **Documentation pages URL** field, enter the URL. 1. Select **Save changes**. +If the "Documentation pages URL" field is empty, the GitLab instance displays a basic version of the documentation sourced from the [`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. + ### Destination requirements When redirecting `/help`, GitLab: - Redirects requests to the specified URL. -- Appends `ee` and the documentation path to the URL. +- Appends `ee` and the documentation path, which includes the version number, to the URL. - Appends `.html` to the URL, and removes `.md` if necessary. For example, if the URL is set to `https://docs.gitlab.com`, requests for `/help/user/admin_area/settings/help_page.md` redirect to: -`https://docs.gitlab.com/ee/user/admin_area/settings/help_page.html`. +`https://docs.gitlab.com/${VERSION}/ee/user/admin_area/settings/help_page.html`. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/import_export_rate_limits.md b/doc/user/admin_area/settings/import_export_rate_limits.md index acf82360042..36a8b340957 100644 --- a/doc/user/admin_area/settings/import_export_rate_limits.md +++ b/doc/user/admin_area/settings/import_export_rate_limits.md @@ -1,7 +1,6 @@ --- -type: reference stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 5a550f15a41..2091191b889 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -66,6 +66,10 @@ The **CI/CD** settings contain: [risks are involved](../../packages/container_registry/reduce_container_registry_storage.md#use-with-external-container-registries) in enabling some of these settings. +## Security and Compliance settings + +- [License compliance settings](security_and_compliance.md#choose-package-registry-metadata-to-sync): Enable or disable synchronization of package metadata by a registry type. + ### Geo **(PREMIUM SELF)** The **Geo** setting contains: @@ -108,8 +112,6 @@ The **Metrics and profiling** settings contain: Enable and configure Grafana. - [Profiling - Performance bar](../../../administration/monitoring/performance/performance_bar.md#enable-the-performance-bar-for-non-administrators) - Enable access to the Performance Bar for non-administrator users in a given group. -- [Self-monitoring](../../../administration/monitoring/gitlab_self_monitoring_project/index.md#create-the-self-monitoring-project) - - Enable or disable instance self-monitoring. - [Usage statistics](usage_statistics.md) - Enable or disable version check and Service Ping. ### Network @@ -130,12 +132,13 @@ The **Network** settings contain: - [Search rate limits](../../../administration/instance_limits.md#search-rate-limit) - Configure global search request rate limits for authenticated and unauthenticated users. - [Deprecated API Rate Limits](deprecated_api_rate_limits.md) - Configure specific limits for deprecated API requests that supersede the user and IP rate limits. -- [Outbound requests](../../../security/webhooks.md) - Allow requests to the local network from hooks and services. +- [Outbound requests](../../../security/webhooks.md) - Allow requests to the local network from webhooks and integrations, or deny all outbound requests. - [Protected Paths](protected_paths.md) - Configure paths to be protected by Rack Attack. - [Incident Management Limits](../../../operations/incident_management/index.md) - Limit the number of inbound alerts that can be sent to a project. - [Notes creation limit](rate_limit_on_notes_creation.md) - Set a rate limit on the note creation requests. - [Get single user limit](rate_limit_on_users_api.md) - Set a rate limit on users API endpoint to get a user by ID. +- [Projects API rate limits for unauthenticated requests](rate_limit_on_projects_api.md) - Set a rate limit on Projects list API endpoint for unauthenticated requests. ### Preferences diff --git a/doc/user/admin_area/settings/project_integration_management.md b/doc/user/admin_area/settings/project_integration_management.md index 1e7c75363ab..dd4349fca2e 100644 --- a/doc/user/admin_area/settings/project_integration_management.md +++ b/doc/user/admin_area/settings/project_integration_management.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/admin_area/settings/rate_limit_on_projects_api.md b/doc/user/admin_area/settings/rate_limit_on_projects_api.md new file mode 100644 index 00000000000..29e72daf579 --- /dev/null +++ b/doc/user/admin_area/settings/rate_limit_on_projects_api.md @@ -0,0 +1,36 @@ +--- +type: reference +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limit on Projects API **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112283) in GitLab 15.10 with a [flag](../../../administration/feature_flags.md) named `rate_limit_for_unauthenticated_projects_api_access`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/391922) on May 08, 2023. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119603) in GitLab 16.0 by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120445) in GitLab 16.0. Feature flag `rate_limit_for_unauthenticated_projects_api_access` removed. + +You can configure the rate limit per IP address for unauthenticated requests to the [list all projects API](../../../api/projects.md#list-all-projects). + +To change the rate limit: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Network**. +1. Expand **Projects API rate limit**. +1. In the **Maximum requests per 10 minutes per IP address** text box, enter the new value. +1. Select **Save changes**. + +The rate limit: + +- Applies per IP address. +- Doesn't apply to authenticated requests. +- Can be set to 0 to disable rate limiting. + +The default value of the rate limit is `400`. + +Requests over the rate limit are logged into the `auth.log` file. + +For example, if you set a limit of 400, unauthenticated requests to the `GET /projects` API endpoint that +exceed a rate of 400 within 10 minutes are blocked. Access to the endpoint is restored after ten minutes have elapsed. diff --git a/doc/user/admin_area/settings/security_and_compliance.md b/doc/user/admin_area/settings/security_and_compliance.md new file mode 100644 index 00000000000..c7f4d6a3ede --- /dev/null +++ b/doc/user/admin_area/settings/security_and_compliance.md @@ -0,0 +1,24 @@ +--- +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Security and Compliance Admin Area settings **(ULTIMATE SELF)** + +The settings for package metadata synchronization are located in the [Admin Area](index.md). + +## Choose package registry metadata to sync + +WARNING: +The full package metadata sync can add up to 30 GB to the PostgreSQL database. Ensure you have provisioned enough disk space for the database before enabling this feature. +We are actively working on reducing this data size in [epic 10415](https://gitlab.com/groups/gitlab-org/-/epics/10415). + +To choose the packages you want to synchronize with the GitLab License Database for [License Compliance](../../compliance/license_scanning_of_cyclonedx_files/index.md): + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > Security and Compliance**. +1. Expand **License Compliance**. +1. Select or clear checkboxes for the package registries that you want to sync. +1. Select **Save changes**. diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index 82a54787101..951e0784c88 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -103,6 +103,8 @@ To turn off Admin Mode for your current session, on the top bar, select **Main m ### Limitations of Admin Mode +Admin Mode times out after six hours, and you cannot change this timeout limit. + The following access methods are **not** protected by Admin Mode: - Git client access (SSH using public keys or HTTPS using Personal Access Tokens). @@ -158,7 +160,7 @@ see [Email notification for unknown sign-ins](../../profile/notifications.md#not All users that are not logged in are redirected to the page represented by the configured **Home page URL** if value is not empty. -All users are redirected to the page represented by the configured **After sign-out path** +All users are redirected to the page represented by the configured **Sign-out page URL** after sign out if value is not empty. In the **Sign-in restrictions** section, scroll to the **Sign-in text** field. You can add a diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index c44901b1ad7..3bf52bfe001 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -51,17 +51,26 @@ signing up using OmniAuth or LDAP, set `block_auto_created_users` to `true` in t [OmniAuth configuration](../../../integration/omniauth.md#configure-common-settings) or [LDAP configuration](../../../administration/auth/ldap/index.md#basic-configuration-settings). -## Require email confirmation +## Confirm user email + +> - Soft email confirmation [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47003) in GitLab 12.2 [with a flag](../../../operations/feature_flags.md) named `soft_email_confirmation`. +> - Soft email confirmation [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107302/diffs) from a feature flag to an application setting in GitLab 15.9. You can send confirmation emails during sign up and require that users confirm their email address before they are allowed to sign in. -To enforce confirmation of the email address used for new sign ups: +For example, to enforce confirmation of the email address used for new sign ups: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. 1. Under **Email confirmation settings**, select **Hard**. +The following settings are available: + +- **Hard** - Send a confirmation email during sign up. New users must confirm their email address before they can log in. +- **Soft** - Send a confirmation email during sign up. New users can log in immediately, but must confirm their email in three days. Unconfirmed accounts are deleted. +- **Off** - New users can sign up without confirming their email address. + ## User cap > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4315) in GitLab 13.7. @@ -95,22 +104,6 @@ New user sign ups are subject to the user cap restriction. New users sign ups are not subject to the user cap restriction. Users in pending approval state are automatically approved in a background job. -## Soft email confirmation - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47003) in GitLab 12.2. -> - It's [deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-soft-email-confirmation). - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -The soft email confirmation improves the sign-up experience for new users by allowing -them to sign in without an immediate confirmation when an email confirmation is required. -GitLab shows the user a reminder to confirm their email address, and the user can't -create or update pipelines until their email address is confirmed. - ## Minimum password length limit > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) in GitLab 12.6 @@ -171,25 +164,6 @@ semicolon, comma, or a new line. ![Domain Denylist](img/domain_denylist_v14_1.png) -### Enable or disable soft email confirmation - -Soft email confirmation is under development but ready for production use. -It is deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:soft_email_confirmation) -``` - -To disable it: - -```ruby -Feature.disable(:soft_email_confirmation) -``` - ## Set up LDAP user filter You can limit GitLab access to a subset of the LDAP users on your LDAP server. diff --git a/doc/user/admin_area/settings/terraform_limits.md b/doc/user/admin_area/settings/terraform_limits.md index 4e54c7a3459..05b1f2d8838 100644 --- a/doc/user/admin_area/settings/terraform_limits.md +++ b/doc/user/admin_area/settings/terraform_limits.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index 4f6e727f673..6037b24a294 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 212769ed89b..ba226e0f05b 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -27,7 +27,7 @@ There are several other benefits to enabling Service Ping: - Analyze the users' activities over time of your GitLab installation. - A [DevOps Score](../analytics/dev_ops_reports.md#devops-score) to give you an overview of your entire instance's adoption of concurrent DevOps from planning to monitoring. -- More proactive support (assuming that our [Customer Success Managers (CSMs)](https://about.gitlab.com/job-families/sales/customer-success-management/) and support organization used the data to deliver more value). +- More proactive support (assuming that our [Customer Success Managers (CSMs)](https://handbook.gitlab.com/job-families/sales/customer-success-management/) and support organization used the data to deliver more value). - Insight and advice into how to get the most value out of your investment in GitLab. - Reports that show how you compare against other similar organizations (anonymized), with specific advice and recommendations on how to improve your DevOps processes. - Participation in our [Registration Features Program](#registration-features-program) to receive free paid features. @@ -50,6 +50,14 @@ tier. Users can continue to access the features in a paid tier without sharing u - [Repository size limit](../settings/account_and_limit_settings.md#repository-size-limit). - [Group access restriction by IP address](../../group/access_and_permissions.md#restrict-group-access-by-ip-address). +### Features available in 16.0 and later + +- [View description change history](../../../user/discussions/index.md#view-description-change-history). +- [Maintenance mode](../../../administration/maintenance_mode/index.md). +- [Configurable issue boards](../../project/issue_board.md#configurable-issue-boards). +- [Coverage-guided fuzz testing](../../application_security/coverage_fuzzing/index.md). +- [Password complexity requirements](../../../user/admin_area/settings/sign_up_restrictions.md#password-complexity-requirements). + NOTE: Registration is not yet required for participation, but may be added in a future milestone. @@ -119,6 +127,11 @@ To enable or disable Service Ping and version check: 1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. 1. Select **Save changes**. +NOTE: +Service Ping settings only control whether the data is being shared with GitLab, or used only internally. +Even if you disable Service Ping, the `gitlab_service_ping_worker` background job still periodically generates a Service Ping payload for your instance. +The payload is available in the [Service Usage data](#manually-upload-service-ping-payload) admin section. + ## Disable usage statistics with the configuration file NOTE: @@ -194,6 +207,11 @@ To upload the payload manually: The uploaded file is encrypted and sent using secure HTTPS protocol. HTTPS creates a secure communication channel between web browser and the server, and protects transmitted data against man-in-the-middle attacks. +If there are problems with the manual upload: + +1. Open a confidential issue in the [security fork of version app project](https://gitlab.com/gitlab-org/security/version.gitlab.com). +1. Attach the JSON payload if possible. +1. Tag `@gitlab-org/analytics-section/product-intelligence` who will triage the issue. <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index acff483e4f8..edcf1a80aca 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -54,6 +54,9 @@ By default both administrators and anyone with the **Owner** role can delete a p > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) from default delayed project deletion in GitLab 15.1. > - [Enabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89466) in GitLab 15.1. > - [Disabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95495) in GitLab 15.3. +> - [Removed option to delete immediately](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. +> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) on May 08, 2023. +> - Enabled delayed deletion by default and removed the option to delete immediately [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. Instance-level protection against accidental deletion of groups and projects. @@ -82,6 +85,7 @@ To configure delayed project deletion: 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. 1. Scroll to: + - (In GitLab 15.11 and later with `always_perform_delayed_deletion` feature flag enabled, or GitLab 16.0 and later) **Deletion protection** and set the retention period to a value between `1` and `90`. - (GitLab 15.1 and later) **Deletion protection** and select keep deleted groups and projects, and select a retention period. - (GitLab 15.0 and earlier) **Default delayed project protection** and select **Enable delayed project deletion by default for newly-created groups.** Then set a retention period in **Default deletion delay**. @@ -98,6 +102,10 @@ In GitLab 15.1, and later this setting is enforced on groups when disabled and i Groups remain restorable if the retention period is `1` or more days. In GitLab 15.1 and later, delayed group deletion can be enabled by setting **Deletion projection** to **Keep deleted**. +In GitLab 15.11 and later with the `always_perform_delayed_deletion` feature flag enabled, or in GitLab 16.0 and later: + +- The **Keep deleted** option is removed. +- Delayed group deletion is the default. ### Override defaults and delete immediately @@ -155,6 +163,10 @@ For more details on group visibility, see ## Restrict visibility levels +When restricting visibility levels, consider how these restrictions interact +with permissions for subgroups and projects that inherit their visibility from +the item you're changing. + To restrict visibility levels for groups, projects, snippets, and selected pages: 1. Sign in to GitLab as a user with Administrator access level. @@ -179,7 +191,8 @@ For more details on project visibility, see ## Configure allowed import sources -You can specify from which hosting sites users can [import their projects](../../project/import/index.md): +Before you can import projects from other systems, you must enable the +[import source](../../gitlab_com/index.md#default-import-sources) for that system. 1. Sign in to GitLab as a user with Administrator access level. 1. On the top bar, select **Main menu > Admin**. @@ -207,9 +220,6 @@ To enable the export of You can enable migration of groups by direct transfer using the UI. -To also migrate projects with the groups, you must enable the -[`bulk_import_projects` feature flag](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). - To enable migration of groups by direct transfer: 1. Sign in to GitLab as a user with Administrator access level. diff --git a/doc/user/admin_area/user_cohorts.md b/doc/user/admin_area/user_cohorts.md index c6d4f0b8e00..b47ae561689 100644 --- a/doc/user/admin_area/user_cohorts.md +++ b/doc/user/admin_area/user_cohorts.md @@ -8,14 +8,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can analyze your users' GitLab activities over time. -To view user cohorts: - -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **Overview > Users**. -1. Select the **Cohorts** tab. - -## Overview - How do you interpret the user cohorts table? Let's review an example with the following user cohorts: @@ -37,3 +29,11 @@ How do we measure the activity of users? GitLab considers a user active if: requests ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54947) in GitLab 11.8). - The user uses the API. - The user uses the GraphQL API. + +## View user cohorts + +To view user cohorts: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Overview > Users**. +1. Select the **Cohorts** tab. diff --git a/doc/user/ai_features.md b/doc/user/ai_features.md new file mode 100644 index 00000000000..7c9480d308f --- /dev/null +++ b/doc/user/ai_features.md @@ -0,0 +1,214 @@ +--- +stage: ModelOps +group: AI Assisted +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference +--- + +# AI/ML powered features + +GitLab is creating AI-assisted features across our DevSecOps platform. These features aim to help increase velocity and solve key pain points across the software development lifecycle. + +## Enable AI/ML features + +> Introduced in GitLab 16.0 and is [actively being rolled out](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118222). + +Prerequisites: + +- You must have the Owner role for the group. + +To enable AI/ML features: + +- Enable the [Experiment features setting](group/manage.md#group-experiment-features-setting). +- The [third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) is enabled by default. To disable AI features powered by third-party APIs, disable this setting. + +These settings give you control over which features are enabled. These settings work together so you can have a mix of both experimental and third-party AI features. + +## Generally Available AI features + +When a feature is [Generally Available](../policy/alpha-beta-support.md#generally-available-ga), it does not require the [group-level Experiment features setting](group/manage.md#group-experiment-features-setting) to be enabled. Some of these features might require the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting). + +The following feature is Generally Available: + +- [Suggested Reviewers](project/merge_requests/reviews/index.md#suggested-reviewers) + +## Beta AI features + +[Beta features](../policy/alpha-beta-support.md#beta) do not require the [group-level Experiment features setting](group/manage.md#group-experiment-features-setting) to be enabled. + +The following feature is in Beta: + +- [Code Suggestions](project/repository/code_suggestions.md) + +## Experiment AI features + +[Experiment features](../policy/alpha-beta-support.md#experiment) will soon require the [group-level Experiment features setting](group/manage.md#group-experiment-features-setting) to be enabled. + +## Third-party AI features + +Third-party AI features require the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) to be enabled. Experiment third-party AI features also require the [Experiment features setting](group/manage.md#group-experiment-features-setting) to be enabled. + +### Explain Selected Code in the Web UI **(ULTIMATE SAAS)** + +> Introduced in GitLab 15.11 as an [Experiment](../policy/alpha-beta-support.md#experiment) on GitLab.com. + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. + +GitLab can help you get up to speed faster if you: + +- Spend a lot of time trying to understand pieces of code that others have created, or +- Struggle to understand code written in a language that you are not familiar with. + +By using a large language model, GitLab can explain the code in natural language. + +Prerequisites: + +- The project must be a public project on GitLab.com. +- You must have the GitLab Ultimate subscription tier. +- You must be a member of the project. + +To explain your code: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**, then select your merge request. +1. On the secondary menu, select **Changes**. +1. Go to the file, and select the lines that you want to have explained. +1. On the left side, select the question mark (**{question}**). You might have to scroll to the first line of your selection to view it. This sends the selected code, together with a prompt, to provide an explanation to the large language model. +1. A drawer is displayed on the right side of the page. Wait a moment for the explanation to be generated. +1. Provide feedback about how satisfied you are with the explanation, so we can improve the results. + +![How to use the Explain Code Experiment](img/explain_code_experiment.png) + +We cannot guarantee that the large language model produces results that are correct. Use the explanation with caution. + +### Explain this Vulnerability in the Web UI **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10368) in GitLab 16.0 as an [Experiment](../policy/alpha-beta-support.md#experiment) on GitLab.com. + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by Google AI. + +GitLab can help you with your vulnerability by using a large language model to: + +- Summarize the vulnerability. +- Help developers and security analysts get started understanding the vulnerability, how it could be exploited, and how to fix it. +- Provide a suggested mitigation. + +Prerequisites: + +- You must have the GitLab Ultimate subscription tier. +- You must be a member of the project. +- The vulnerability must be a SAST finding. + +To explain your vulnerability: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. +1. Find a SAST vulnerability. +1. Open the vulnerability. +1. Select **Try it out**. + +Review the drawer on the right-hand side of your screen. + +![How to use Explain this Vulnerability Experiment](img/explain_this_vulnerability.png) + +We cannot guarantee that the large language model produces results that are correct. Use the explanation with caution. + +### GitLab Chat **(ULTIMATE SAAS)** + +> Introduced in GitLab 16.0 as an [Experiment](../policy/alpha-beta-support.md#experiment). + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) to be enabled. + +Getting help has never been easier. If you have a question about how the GitLab product works, you can ask product how-to questions and get AI generated support from GitLab Chat. + +1. In the lower-left corner, select the Help icon. +1. Select **Ask in GitLab Chat**. A drawer opens on the right side of your screen. +1. Enter your question in the chat input box and press **Enter** or select **Send**. It may take a few seconds for the interactive AI chat to search the product documentation and produce an answer. + +To give feedback, select the **Give Feedback** link. + +### Summarize merge request changes **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10400) in GitLab 16.0 as an [Experiment](../policy/alpha-beta-support.md#experiment). + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) to be enabled. + +You can generate a merge request summary by using the `/summarize_diff` quick action in a merge request comment. This action posts a comment from a GitLab bot. The comment provides a summary of the changes and the related SHA for when that summary was generated. + +Provide feedback on this experimental feature in [issue 408726](https://gitlab.com/gitlab-org/gitlab/-/issues/408726). + +**Data usage**: When you use this quick action, the diff of changes between the head of the source branch +and the target branch is sent to the large language model referenced above. + +### Summarize my merge request review **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10466) in GitLab 16.0 as an [Experiment](../policy/alpha-beta-support.md#experiment). + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) to be enabled. + +When you've completed your review of a merge request and are ready to [submit your review](project/merge_requests/reviews/index.md#submit-a-review) you can choose to have summary generated for you. To generate the summary: + +1. Select the AI Actions dropdown list. +1. Select **Summarize my code review**. + +The summary is generated and entered in to the comment box where you can edit and refine prior to submitting with your review. + +Provide feedback on this experimental feature in [issue 408991](https://gitlab.com/gitlab-org/gitlab/-/issues/408991). + +**Data usage**: When you use this feature, the following data is sent to the large language model referenced above: + +- Draft comment's text +- File path of the commented files + +### Generate suggested tests in merge requests **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10366) in GitLab 16.0 as an [Experiment](../policy/alpha-beta-support.md#experiment). + +This feature is an [Experiment](../policy/alpha-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting) to be enabled. + +When in a merge request you can choose to have GitLab suggest tests for the file you are reviewing. This can help to determine if appropriate test coverage has been provided or help with writing tests to provide more coverage for your project. To generate a test suggestion: + +1. Select the menu icon on the header of a file. +1. Select **Generate test with AI**. + +A sidebar opens where the test suggestion is generated. From there you can choose to copy that suggestion in to your editor as the start of your tests. + +Feedback on this experimental feature can be provided in [issue 408995](https://gitlab.com/gitlab-org/gitlab/-/issues/408995). + +**Data usage**: When you use this feature, the following data is sent to the large language model referenced above: + +- Contents of the file +- The file name + +## Data Usage + +GitLab AI features leverage generative AI to help increase velocity and aim to help make you more productive. Each feature operates independently of other features and is not required for other features to function. + +### Progressive enhancement + +These features are designed as a progressive enhancement to existing GitLab features across our DevSecOps platform. They are designed to fail gracefully and should not prevent the core functionality of the underlying feature. Please note each feature is subject to its expected functionality as defined by the relevant [feature support policy](../policy/alpha-beta-support.md). + +### Stability and performance + +These features are in a variety of [feature support levels](../policy/alpha-beta-support.md#beta). Due to the nature of these features, there may be high demand for usage which may cause degraded performance or unexpected downtime of the feature. We have built these features to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable **beta and experimental** features for any or all customers at any time at our discretion. + +## Third party services + +### Data privacy + +Some AI features require the use of third-party AI services models and APIs from: Google AI and OpenAI. The processing of any personal data is in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). You may also visit the [Sub-Processors page](https://about.gitlab.com/privacy/subprocessors/#third-party-sub-processors) to see the list of our Sub-Processors that we use in order to provide these features. + +Group owners can control which top-level groups have access to third-party AI features by using the [group level third-party AI features setting](group/manage.md#group-third-party-ai-features-setting). + +### Model accuracy and quality + +Generative AI may produce unexpected results that may be: + +- Low-quality +- Incoherent +- Incomplete +- Produce failed pipelines +- Insecure code +- Offensive or insensitive + +GitLab is actively iterating on all our AI-assisted capabilities to improve the quality of the generated content. We will continue improving the quality through prompt engineering, evaluating new AI/ML models to power these features, and through novel heuristics built into these features directly. diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index ef2d3a76990..bde7fdc6c2d 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -39,7 +39,7 @@ To view CI/CD analytics: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.8. -The deployment frequency charts show information about the deployment +The [deployment frequency](dora_metrics.md#deployment-frequency) charts show information about the deployment frequency to the `production` environment. The environment must be part of the [production deployment tier](../../ci/environments/index.md#deployment-tier-of-environments) for its deployment information to appear on the graphs. @@ -60,7 +60,7 @@ To view the deployment frequency chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250329) in GitLab 13.11. -The lead time for changes chart shows information about how long it takes for +The [lead time for changes](dora_metrics.md#lead-time-for-changes) chart shows information about how long it takes for merge requests to be deployed to a production environment. This chart is available for groups and projects. - Small lead times indicate fast, efficient deployment @@ -82,7 +82,7 @@ To view the lead time for changes chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356959) in GitLab 15.1 -The time to restore service chart shows information about the median time an incident was open in a production environment. This chart is available for groups and projects. +The [time to restore service](dora_metrics.md#time-to-restore-service) chart shows information about the median time an incident was open in a production environment. This chart is available for groups and projects. Time to restore service is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery. @@ -98,7 +98,7 @@ To view the time to restore service chart: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357072) in GitLab 15.2 -The change failure rate chart shows information about the percentage of deployments that cause an incident in a production environment. This chart is available for groups and projects. +The [change failure rate](dora_metrics.md#change-failure-rate) chart shows information about the percentage of deployments that cause an incident in a production environment. This chart is available for groups and projects. Change failure rate is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery. diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index fae9545003f..2e76252f7d3 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -5,7 +5,6 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- - # Code review analytics **(PREMIUM)** > Moved to GitLab Premium in 13.9. @@ -24,6 +23,9 @@ and improve your code review process. - Opportunities to accelerate your development cycle. - Few comments and approvers may indicate a lack of available team members. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video explanation, see [Code review analytics: Faster code review](https://www.youtube.com/watch?v=OkLaWhYkASM). + ## View code review analytics Prerequisite: diff --git a/doc/user/analytics/dora_metrics.md b/doc/user/analytics/dora_metrics.md index 9ac949f05b4..6b7b1d87843 100644 --- a/doc/user/analytics/dora_metrics.md +++ b/doc/user/analytics/dora_metrics.md @@ -15,21 +15,17 @@ Using these metrics helps improve DevOps efficiency and communicate performance DORA includes four key metrics, divided into two core areas of DevOps: -- [Deployment Frequency](#deployment-frequency) and [Lead Time for Change](#lead-time-for-changes) measure team velocity. -- [Change Failure Rate](#change-failure-rate) and [Time to Restore Service](#time-to-restore-service) measure stability. +- [Deployment frequency](#deployment-frequency) and [Lead time for changes](#lead-time-for-changes) measure team velocity. +- [Change failure rate](#change-failure-rate) and [Time to restore service](#time-to-restore-service) measure stability. For software leaders, tracking velocity alongside quality metrics ensures they're not sacrificing quality for speed. -<div class="video-fallback"> - For an overview, see <a href="https://www.youtube.com/watch?v=1BrcMV6rCDw">GitLab Speed Run: DORA metrics in GitLab One DevOps Platform</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/1BrcMV6rCDw" frameborder="0" allowfullscreen> </iframe> -</figure> +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video explanation, see [DORA metrics: User analytics](https://www.youtube.com/watch?v=lM_FbVYuN8s) and [GitLab speed run: DORA metrics](https://www.youtube.com/watch?v=1BrcMV6rCDw). -## DORA Metrics dashboard in Value Stream Analytics +## DORA metrics in Value Stream Analytics -The four DORA metrics are available out-of-the-box in the [Value Stream Analytics (VSA) overview dashboard](../group/value_stream_analytics/index.md#view-dora-metrics-and-key-metrics-for-a-group). +The four DORA metrics are available out-of-the-box in the [Value Streams Dashboard](value_streams_dashboard.md). This helps you visualize the engineering work in the context of end-to-end value delivery. The One DevOps Platform [Value Stream Management](https://gitlab.com/gitlab-org/gitlab/-/value_stream_analytics) provides end-to-end visibility to the entire software delivery lifecycle. @@ -37,76 +33,120 @@ This enables teams and managers to understand all aspects of productivity, quali ## Deployment frequency -Deployment frequency is the frequency of successful deployments to production (hourly, daily, weekly, monthly, or yearly). -This measures how often you deliver value to end users. A higher deployment frequency means you can -get feedback sooner and iterate faster to deliver improvements and features. GitLab measures this as the number of -deployments to a production environment in the given time period. +> [Fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/394712) the frequency calculation formula for the `all` and `monthly` intervals in GitLab 16.0. -Deployment frequency displays in several charts: +Deployment frequency is the frequency of successful deployments to production over the given date range (hourly, daily, weekly, monthly, or yearly). -- [Group-level value stream analytics](../group/value_stream_analytics/index.md) -- [Project-level value stream analytics](value_stream_analytics.md) -- [CI/CD analytics](ci_cd_analytics.md) +Software leaders can use the deployment frequency metric to understand how often the team successfully deploys software to production, and how quickly the teams can respond to customers' requests or new market opportunities. +High deployment frequency means you can get feedback sooner and iterate faster to deliver improvements and features. -To retrieve metrics for deployment frequency, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. +### How deployment frequency is calculated + +In GitLab, Deployment frequency is measured by the average number of deployments per day to a given environment, based on the deployment's end time (its `finished_at` property). +GitLab calculates the deployment frequency from the number of finished deployments on the given day. + +The calculation takes into account the production `environment tier` or the environments named `production/prod`. The environment must be part of the production deployment tier for its deployment information to appear on the graphs. + +### How to improve deployment frequency + +The first step is to benchmark the cadence of code releases between groups and projects. Next, you should consider: + +- Add more automated testing. +- Add more automated code validation. +- Break the changes down into smaller iterations. ## Lead time for changes -DORA Lead time for changes measures the time to successfully deliver a commit into production. -This metric reflects the efficiency of CI/CD pipelines. +Lead time for changes is the amount of time it takes a code change to get into production. + +"Lead time for changes" is not the same as "Lead time". In the value stream, "Lead time" measures the time it takes for work on an issue to move from the moment it's requested (Issue created) to the moment it's fulfilled and delivered (Issue closed). -In GitLab, Lead time for changes calculates the median time it takes for a merge request to get merged into production. -We measure **from** code committed **to** code successfully running in production, without adding the `coding_time` to the calculation. +For software leaders, Lead time for changes reflects the efficiency of CI/CD pipelines and visualizes how quickly work is delivered to customers. +Over time, the lead time for changes should decrease, while your team's performance should increase. Low lead time for changes means more efficient CI/CD pipelines. +In GitLab, Lead time for changes is measure by the `Median time it takes for a merge request to get merged into production (from master)`. -Over time, the lead time for changes should decrease, while your team's performance should increase. +### How lead time for changes is calculated -Lead time for changes displays in several charts: +GitLab calculates Lead time for changes base on the number of seconds to successfully deliver a commit into production - **from** code committed **to** code successfully running in production, without adding the `coding_time` to the calculation. -- [Group-level value stream analytics](../group/value_stream_analytics/index.md) -- [Project-level value stream analytics](value_stream_analytics.md) -- [CI/CD analytics](ci_cd_analytics.md) +### How to improve lead time for changes -To retrieve metrics for lead time for changes, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. +The first step is to benchmark the CI/CD pipelines' efficiency between groups and projects. Next, you should consider: -- The definition of lead time for change can vary widely, which often creates confusion within the industry. -- "Lead time for changes" is not the same as "Lead time". In the value stream, "Lead time" measures the time it takes for work on an issue to move from the moment it's requested (Issue created) to the moment it's fulfilled and delivered (Issue closed). +- Using Value Stream Analytics to identify bottlenecks in the processes. +- Breaking the changes down into smaller iterations. +- Adding more automation. ## Time to restore service -Time to restore service measures how long it takes an organization to recover from a failure in production. -GitLab measures this as the average time required to close the incidents -in the given time period. This assumes: +Time to restore service is the amount of time it takes an organization to recover from a failure in production. + +For software leaders, Time to restore service reflects how long it takes an organization to recover from a failure in production. +Low Time to restore service means the organization can take risks with new innovative features to drive competitive advantages and increase business results. +### How time to restore service is calculated + +In GitLab, Time to restore service is measured as the median time an incident was open for on a production environment. +GitLab calculates the number of seconds an incident was open on a production environment in the given time period. This assumes: + +- [GitLab incidents](../../operations/incident_management/incidents.md) are tracked. - All incidents are related to a production environment. -- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only -one production deployment, and any production deployment is related to no more than one incident). +- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only one production deployment, and any production deployment is related to no more than one incident. -Time to restore service displays in several charts: +### How to improve time to restore service -- [Group-level value stream analytics](../group/value_stream_analytics/index.md) -- [Project-level value stream analytics](value_stream_analytics.md) -- [CI/CD analytics](ci_cd_analytics.md) +The first step is to benchmark the team response and recover from service interruptions and outages, between groups and projects. Next, you should consider: -To retrieve metrics for time to restore service, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. +- Improving the observability into the production environment. +- Improving response workflows. ## Change failure rate -Change failure rate measures the percentage of deployments that cause a failure in production. GitLab measures this as the number -of incidents divided by the number of deployments to a -production environment in the given time period. This assumes: +Change failure rate is how often a change cause failure in production. + +Software leaders can use the change failure rate metric to gain insights into the quality of the code being shipped. +High change failure rate may indicate an inefficient deployment process or insufficient automated testing coverage. + +### How change failure rate is calculated +In GitLab, Change failure rate is measured as the percentage of deployments that cause an incident in production in the given time period. +GitLab calculates this by the number of incidents divided by the number of deployments to a production environment. This assumes: + +- [GitLab incidents](../../operations/incident_management/incidents.md) are tracked. - All incidents are related to a production environment. -- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only -one production deployment, and any production deployment is related to no +- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only one production deployment, and any production deployment is related to no more than one incident. -To retrieve metrics for change failure rate, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. +### How to improve change failure rate + +The first step is to benchmark the quality and stability, between groups and projects. + +To improve this metric, you should consider: -### Insights: Custom DORA reporting +- Finding the right balance between stability and throughput (Deployment frequency and Lead time for changes), and not sacrificing quality for speed. +- Improving the efficacy of code review processes. +- Adding more automated testing. -Custom charts to visualize DORA data with [Insights YAML-based reports](../../user/project/insights/index.md#dora-query-parameters). +## DORA metrics in GitLab -With this new visualization, software leaders can track metrics improvements, understand patterns in their metrics trends, and compare performance between groups and projects. +The DORA metrics are displayed on the following charts: + +- [Value Streams Dashboard](value_streams_dashboard.md), which helps you identify trends, patterns, and opportunities for improvement. +- [CI/CD analytics charts](ci_cd_analytics.md), which show pipeline success rates and duration, and the history of DORA metrics over time. +- Insights reports for [groups](../group/insights/index.md) and [projects](value_stream_analytics.md), where you can also use [DORA query parameters](../../user/project/insights/index.md#dora-query-parameters) to create custom charts. + +The table below provides an overview of the DORA metrics' data aggregation in different charts. + +| Metric name | Measured values | Data aggregation in the [Value Streams Dashboard](value_streams_dashboard.md) | Data aggregation in [CI/CD analytics charts](ci_cd_analytics.md) | Data aggregation in [Custom insights reporting](../../user/project/insights/index.md#dora-query-parameters) | +|---------------------------|-------------------|-----------------------------------------------------|------------------------|----------| +| Deployment frequency | Number of successful deployments | daily average per month | daily average | `day` (default) or `month` | +| Lead time for changes | Number of seconds to successfully deliver a commit into production | daily median per month | median time | `day` (default) or `month` | +| Time to restore service | Number of seconds an incident was open for | daily median per month | daily median | `day` (default) or `month` | +| Change failure rate | percentage of deployments that cause an incident in production | daily median per month | percentage of failed deployments | `day` (default) or `month` | + +## Retrieve DORA metrics data + +To retrieve DORA data, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs. ### Measure DORA metrics without using GitLab CI/CD pipelines @@ -115,6 +155,26 @@ These deployment records are not created for pull-based deployments, for example To track DORA metrics in these cases, you can [create a deployment record](../../api/deployments.md#create-a-deployment) using the Deployments API. See also the documentation page for [Track deployments of an external deployment tool](../../ci/environments/external_deployment_tools.md). +### Measure DORA metrics with Jira + +- Deployment frequency and Lead time for changes are calculated based on GitLab CI/CD and Merge Requests (MRs), and do not require Jira data. +- Time to restore service and Change failure rate require GitLab incidents for the calculation. For more information, see [Measure DORA Time to restore service and Change failure rate with external incidents](#measure-dora-time-to-restore-service-and-change-failure-rate-with-external-incidents). + +### Measure DORA Time to restore service and Change failure rate with external incidents + +[Time to restore service](#time-to-restore-service) and [Change failure rate](#change-failure-rate) +require [GitLab incidents](../../operations/incident_management/manage_incidents.md) to calculate the metrics. + +For PagerDuty, you can set up a [webhook to automatically create a GitLab incident for each PagerDuty incident](../../operations/incident_management/manage_incidents.md#using-the-pagerduty-webhook). +This configuration requires you to make changes in both PagerDuty and GitLab. + +For others incident management tools, you can set up the +[HTTP integration](../../operations/incident_management/integrations.md#http-endpoints), +and use it to automatically: + +1. [Create an incident when an alert is triggered](../../operations/incident_management/manage_incidents.md#automatically-when-an-alert-is-triggered). +1. [Close incidents via recovery alerts](../../operations/incident_management/manage_incidents.md#automatically-close-incidents-via-recovery-alerts). + ### Supported DORA metrics in GitLab | Metric | Level | API | UI chart | Comments | diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index 774063810f3..44af9dc9dea 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -59,7 +59,7 @@ You can use the following analytics features to analyze and visualize the perfor - [Value stream analytics for groups](../group/value_stream_analytics/index.md) - [Value streams dashboard](value_streams_dashboard.md) -## Definitions +## Glossary We use the following terms to describe GitLab analytics: diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 093266e8aee..292a6f430d9 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -1,246 +1,13 @@ --- -stage: Plan -group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-07-22' +redirect_to: '../group/value_stream_analytics/index.md' --- # Value stream analytics for projects **(FREE)** -> - Introduced as cycle analytics prior to GitLab 12.3 at the project level. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12077) in GitLab Premium 12.3 at the group level. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23427) from cycle analytics to value stream analytics in GitLab 12.8. +This document was moved to [another location](../group/value_stream_analytics/index.md). -Value stream analytics provides metrics about each stage of your software development process. - -A **value stream** is the entire work process that delivers value to customers. For example, -the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts -with the Manage stage and ends with the Protect stage. - -Use value stream analytics to identify: - -- The amount of time it takes to go from an idea to production. -- The velocity of a given project. -- Bottlenecks in the development process. -- Factors that cause your software development lifecycle to slow down. - -Value stream analytics is also available for [groups](../group/value_stream_analytics). - -## View value stream analytics - -> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326701) in GitLab 14.3 -> - Sorting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335974) in GitLab 14.4. - -To view value stream analytics for your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. To view metrics for a particular stage, select a stage below the **Filter results** text box. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -1. Optional. Sort results by ascending or descending: - - To sort by most recent or oldest workflow item, select the **Last event** header. - - To sort by most or least amount of time spent in each stage, select the **Duration** header. - -The table shows a list of related workflow items for the selected stage. Based on the stage you choose, this can be: - -- CI/CD jobs -- Issues -- Merge requests -- Pipelines - -A badge next to the workflow items table header shows the number of workflow items that completed the selected stage. - -## View time spent in each development stage - -Value stream analytics shows the median time spent by issues or merge requests in each development stage. - -To view the median time spent in each stage: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -1. To view the median time for each stage, above the **Filter results** text box, point to a stage. - -## View the lead time and cycle time for issues - -Value stream analytics shows the lead time and cycle time for issues in your project: - -- Lead time: Median time from when the issue was created to when it was closed. -- Cycle time: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. - -To view the lead time and cycle time for issues: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Lead Time** and **Cycle Time** metrics display below the **Filter results** text box. - -## View lead time for changes for merge requests **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. - -Lead time for changes is the median duration between when a merge request is merged and when it's deployed to production. - -To view the lead time for changes for merge requests in your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Lead Time for Changes** metrics display below the **Filter results** text box. - -## View number of successful deployments **(FREE)** - -Prerequisites: - -- To view deployment metrics, you must have a -[production environment configured](../../ci/environments/index.md#deployment-tier-of-environments). - -Value stream analytics shows the following deployment metrics for your project within the specified date range: - -- Deploys: The number of successful deployments in the date range. -- Deployment Frequency: The average number of successful deployments per day in the date range. - -If you have a GitLab Premium or Ultimate subscription: - -- The number of successful deployments is calculated with DORA data. -- The data is filtered based on environment and environment tier. - -To view deployment metrics for your project: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. - -The **Deploys** and **Deployment Frequency** metrics display below the **Filter results** text box. - -Deployment metrics are calculated based on data from the -[DORA API](../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). - -NOTE: -In GitLab 13.9 and later, metrics are calculated based on when the deployment was finished. -In GitLab 13.8 and earlier, metrics are calculated based on when the deployment was created. - -## Access permissions for value stream analytics - -Access permissions for value stream analytics depend on the project type. - -| Project type | Permissions | -|--------------|----------------------------------------| -| Public | Anyone can access. | -| Internal | Any authenticated user can access. | -| Private | Any member Guest and above can access. | - -## How value stream analytics measures each stage - -Value stream analytics uses start and end events to measure the time that an issue or merge request -spends in each stage. - -For example, a stage might start when a user adds a label to an issue, and ends when they add another label. -Items aren't included in the stage time calculation if they have not reached the end event. - -| Stage | Measurement method | -|---------|----------------------| -| Issue | The median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone. The label is tracked only if it already includes an [issue board list](../project/issue_board.md) that has been created for the label. | -| Plan | The median time between the action you took for the previous stage, and when you push the first commit to the branch. The first branch commit triggers the transition from **Plan** to **Code**, and at least one of the commits in the branch must include the related issue number (such as `#42`). If the issue number is not included in a commit, that data is not included in the measurement time of the stage. | -| Code | The median time between pushing a first commit (previous stage) and creating a merge request. The process is tracked with the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) in the description of the merge request. For example, if the issue is closed with `Closes #xxx`, `xxx` is the issue number for the merge request. If there is no closing pattern, the start time is set to the create time of the first commit. | -| Test | The time from start to finish for all pipelines. Measures the median time to run the entire pipeline for that project. Related to the time required by GitLab CI/CD to run every job for the commits pushed to that merge request, as defined in the previous stage. | -| Review | The median time taken to review merge requests with a closing issue pattern, from creation to merge. | -| Staging | The median time between merging the merge request (with a closing issue pattern) to the first deployment to a [production environment](../../ci/environments/index.md#deployment-tier-of-environments). Data is not collected without a production environment. | - -## Example workflow - -This example shows a workflow through all seven stages in one day. In this -example, milestones have been created and CI for testing and setting environments is configured. - -- 09:00: Create issue. **Issue** stage starts. -- 11:00: Add issue to a milestone, start work on the issue, and create a branch locally. -**Issue** stage stops and **Plan** stage starts. -- 12:00: Make the first commit. -- 12:30: Make the second commit to the branch that mentions the issue number. **Plan** stage stops and **Code** stage starts. -- 14:00: Push branch and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically). **Code** stage stops and **Test** and **Review** stages start. -- The CI takes 5 minutes to run scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/index.md). -**Test** stage stops. -- Review merge request. -- 19:00: Merge the merge request. **Review** stage stops and **Staging** stage starts. -- 19:30: Deployment to the `production` environment starts and finishes. **Staging** stops. - -Value stream analytics records the following times for each stage: - -- **Issue**: 09:00 to 11:00: 2 hrs -- **Plan**: 11:00 to 12:00: 1 hr -- **Code**: 12:00 to 14:00: 2 hrs -- **Test**: 5 minutes -- **Review**: 14:00 to 19:00: 5 hrs -- **Staging**: 19:00 to 19:30: 30 minutes - -Keep in mind the following observations related to this example: - -- Although this example specifies the issue number in a later commit, the process -still collects analytics data for the issue. -- The time required in the **Test** stage is included in the **Review** process, -as every merge request should be tested. -- This example illustrates only one cycle of multiple stages. The value -stream analytics dashboard shows the calculated median elapsed time for these issues. -- Value stream analytics identifies production environments based on the -[deployment tier of environments](../../ci/environments/index.md#deployment-tier-of-environments). - -## Troubleshooting - -### 100% CPU utilization by Sidekiq `cronjob:analytics_cycle_analytics` - -It is possible that Value stream analytics background jobs -strongly impact performance by monopolizing CPU resources. - -To recover from this situation: - -1. Disable the feature for all projects in [the Rails console](../../administration/operations/rails_console.md), - and remove existing jobs: - - ```ruby - Project.find_each do |p| - p.analytics_access_level='disabled'; - p.save! - end - - Analytics::CycleAnalytics::GroupStage.delete_all - Analytics::CycleAnalytics::Aggregation.delete_all - ``` - -1. Configure a [Sidekiq routing](../../administration/sidekiq/processing_specific_job_classes.md) - with for example a single `feature_category=value_stream_management` - and multiple `feature_category!=value_stream_management` entries. - Find other relevant queue metadata in the - [Enterprise Edition list](../../administration/sidekiq/processing_specific_job_classes.md#list-of-available-job-classes). -1. Enable value stream analytics for one project after another. - You might need to tweak the Sidekiq routing further according to your performance requirements. +<!-- This redirect file can be deleted after 2023-07-22. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/analytics/value_streams_dashboard.md b/doc/user/analytics/value_streams_dashboard.md index f8cfb1bf06b..384f4ce3455 100644 --- a/doc/user/analytics/value_streams_dashboard.md +++ b/doc/user/analytics/value_streams_dashboard.md @@ -4,21 +4,18 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Value Streams Dashboard **(PREMIUM)** +# Value Streams Dashboard (Beta) **(ULTIMATE)** -> Introduced in GitLab 15.8 as a Closed [Beta](../../policy/alpha-beta-support.md#beta-features) feature. +> - Introduced in GitLab 15.8 as a Closed [Beta](../../policy/alpha-beta-support.md#beta) feature [with a flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. Disabled by default. +> - Released in GitLab 15.11 as an Open [Beta](../../policy/alpha-beta-support.md#beta) feature [with a flag](../../administration/feature_flags.md) named `group_analytics_dashboards_page`. Enabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/392734) in GitLab 16.0. Feature flag `group_analytics_dashboards_page` removed. You can leave feedback on dashboard bugs or functionality in [issue 381787](https://gitlab.com/gitlab-org/gitlab/-/issues/381787). -This feature is not ready for production use. - -The Value Streams Dashboard is a customizable dashboard to enable decision-makers to identify trends, patterns, and opportunities for digital transformation improvements. +The Value Streams Dashboard is a customizable dashboard that enables decision-makers to identify trends, patterns, and opportunities for digital transformation improvements. This page is a work in progress, and we're updating the information as we add more features. For more information, see the [Value Stream Management category direction page](https://about.gitlab.com/direction/plan/value_stream_management/). -After the feature flag is enabled, to open the new page, append this path `/analytics/dashboards` to the group URL -(for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards`). - ## Initial use case Our initial use case is focused on providing the ability to compare software delivery metrics. @@ -27,7 +24,7 @@ This comparison can help decision-makers understand whether projects and groups The beta version of the Value Streams Dashboard includes the following metrics: - [DORA metrics](dora_metrics.md) -- [Value Stream Analytics (VSA) - flow metrics](value_stream_analytics.md) +- [Value Stream Analytics (VSA) - flow metrics](../group/value_stream_analytics/index.md) The Value Streams Dashboard allows you to: @@ -49,20 +46,91 @@ that are the largest value contributors, overperforming, or underperforming. You can also drill down the metrics for further analysis. When you hover over a metric, a tooltip displays an explanation of the metric and a link to the related documentation page. -## Customize the dashboard widgets +## View the value streams dashboard + +Prerequisite: + +- To view the value streams dashboard for a group, you must have at least the Reporter role for the group. + +To view the value streams dashboard: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, in the **Key metrics** row, select **Value Streams Dashboard / DORA**. +1. Optional. To open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL +(for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). + +## Customize the dashboard panels You can customize the Value Streams Dashboard and configure what subgroups and projects to include in the page. A view can display maximum four subgroups or projects. +### Using query parameters + To display multiple subgroups and projects, specify their path as a URL parameter. -For example, the parameter `query=gitlab-org/gitlab-foss,gitlab-org/gitlab,gitlab-org/gitlab-design,gitlab-org/gitlab-docs` displays three separate widgets, one each for the: +For example, the parameter `query=gitlab-org/gitlab-ui,gitlab-org/plan-stage` displays three separate panels, one each for the: - `gitlab-org` group - `gitlab-ui` project - `gitlab-org/plan-stage` subgroup +### Using YAML configuration + +To change the default content of the page, you need to create a YAML configuration file in a project of your choice. Query parameters can still be used to override the YAML configuration. + +First, you need to set up the project. + +Prerequisite: + +- You must have at least the Maintainer role for the group. + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Scroll to **Analytics Dashboards** and select **Expand**. +1. Select the project where you would like to store your YAML configuration file. +1. Select **Save changes**. + +After you have set up the project, set up the configuration file: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. In the default branch, create the configuration file: `.gitlab/analytics/dashboards/value_streams/value_streams.yaml`. +1. In the `value_streams.yaml` configuration file, fill in the configuration options: + +```yaml +# title - Change the title of the Value Streams Dashboard. [optional] +title: 'Custom Dashboard title' + +# description - Change the description of the Value Streams Dashboard. [optional] +description: 'Custom description' + +# panels - List of panels that contain panel settings. +# title - Change the title of the panel. [optional] +# data.namespace - The Group or Project path to use for the chart panel. +panels: + - title: 'My Custom Project' + data: + namespace: group/my-custom-project + - data: + namespace: group/another-project + - title: 'My Custom Group' + data: + namespace: group/my-custom-group + - data: + namespace: group/another-group +``` + + The following example has an option configuration for a panel for the `my-group` namespace: + + ```yaml + panels: + - data: + namespace: my-group + ``` + ## Dashboard metrics and drill-down reports | Metric | Description | Drill-down report | Documentation page | @@ -71,7 +139,9 @@ For example, the parameter `query=gitlab-org/gitlab-foss,gitlab-org/gitlab,gitla | Lead time for changes | The time to successfully deliver a commit into production. This metric reflects the efficiency of CI/CD pipelines. | [Lead time tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=lead-time) | [Lead time for changes](dora_metrics.md#lead-time-for-changes) | | Time to restore service | The time it takes an organization to recover from a failure in production. | [Time to restore service tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=time-to-restore-service) | [Time to restore service](dora_metrics.md#time-to-restore-service) | | Change failure rate | Percentage of deployments that cause an incident in production. | [Change failure rate tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=change-failure-rate) | [Change failure rate](dora_metrics.md#change-failure-rate) | -| VSA Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](value_stream_analytics.md#view-the-lead-time-and-cycle-time-for-issues) | -| VSA Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](value_stream_analytics.md#view-the-lead-time-and-cycle-time-for-issues) | +| Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | +| Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | | New issues | Number of new issues created. | [Issue Analytics](https://gitlab.com/groups/gitlab-org/-/issues_analytics) | Issue analytics [for projects](issue_analytics.md) and [for groups](../../user/group/issues_analytics/index.md) | | Number of deploys | Total number of deploys to production. | [Merge Request Analytics](https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics) | [Merge request analytics](merge_request_analytics.md) | +| Critical vulnerabilities over time | Critical vulnerabilities over time in project or group | [Vulnerability report](https://gitlab.com/gitlab-org/gitlab/-/security/vulnerability_report) | [Vulnerability report](../application_security/vulnerability_report/index.md) | +| High vulnerabilities over time | High vulnerabilities over time in project or group | [Vulnerability report](https://gitlab.com/gitlab-org/gitlab/-/security/vulnerability_report) | [Vulnerability report](../application_security/vulnerability_report/index.md) | diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index b55c5a1b299..b613b0cc33e 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -16,6 +16,9 @@ We recommend that you use fuzz testing in addition to [GitLab Secure](../index.m other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run fuzz tests as part your CI/CD workflow. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Web API Fuzzing](https://www.youtube.com/watch?v=oUHsfvLGhDk). + ## When Web API fuzzing runs Web API fuzzing runs in the `fuzz` stage of the CI/CD pipeline. To ensure API fuzzing scans the @@ -50,6 +53,7 @@ Example projects using these methods are available: - [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/postman-api-fuzzing-example) - [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/graphql-api-fuzzing-example) - [Example SOAP project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/soap-api-fuzzing-example) +- [Authentication Token using Selenium](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/auth-token-selenium) ## Enable Web API fuzzing @@ -98,7 +102,7 @@ a YAML snippet that you can paste in your GitLab CI/CD configuration. To generate an API Fuzzing configuration snippet: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **API Fuzzing** row, select **Enable API Fuzzing**. 1. Complete the fields. For details see [Available CI/CD variables](#available-cicd-variables). 1. Select **Generate code snippet**. @@ -330,6 +334,8 @@ This example is a minimal configuration for API Fuzzing. From here you can: #### API Fuzzing with a GraphQL Schema file +API Fuzzing can use a GraphQL schema file to understand and test a GraphQL endpoint that has introspection disabled. To use a GraphQL schema file, it must be in the introspection JSON format. A GraphQL schema can be converted to a the introspection JSON format using an online 3rd party tool: [https://transform.tools/graphql-to-introspection-json](https://transform.tools/graphql-to-introspection-json). + To configure API Fuzzing to use a GraphQl schema file that provides information about the target API to test: 1. [Include](../../../ci/yaml/index.md#includetemplate) @@ -1994,7 +2000,7 @@ When configured correctly, a CI/CD pipeline contains a `fuzz` stage and an `apif typical operation, the job always succeeds even if faults are identified during fuzz testing. Faults are displayed on the **Security** pipeline tab with the suite name. When testing against the -repositories default branch, the fuzzing faults are also shown on the Security & Compliance's +repositories default branch, the fuzzing faults are also shown on the Security and Compliance's Vulnerability Report page. To prevent an excessive number of reported faults, the API fuzzing scanner limits the number of @@ -2026,7 +2032,7 @@ Follow these steps to view details of a fuzzing fault: 1. You can view faults in a project, or a merge request: - - In a project, go to the project's **Security & Compliance > Vulnerability Report** + - In a project, go to the project's **Security and Compliance > Vulnerability Report** page. This page shows all vulnerabilities from the default branch only. - In a merge request, go the merge request's **Security** section and select the **Expand** button. API Fuzzing faults are available in a section labeled @@ -2345,9 +2351,12 @@ apifuzzer_v1: FUZZAPI_EXCLUDE_PATHS: /api/v1/** rules: rules: - - if: $API_FUZZING_DISABLED + - if: $API_FUZZING_DISABLED == 'true' || $API_FUZZING_DISABLED == '1' when: never - - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH && + - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH == 'true' && + $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME + when: never + - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH == '1' && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - if: $CI_COMMIT_BRANCH && @@ -2361,7 +2370,7 @@ apifuzzer_v2: FUZZAPI_EXCLUDE_PATHS: /api/v2/** rules: rules: - - if: $API_FUZZING_DISABLED + - if: $API_FUZZING_DISABLED == 'true' || $API_FUZZING_DISABLED == '1' when: never - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME @@ -2396,7 +2405,7 @@ apifuzzer_branch: FUZZAPI_EXCLUDE_PATHS: /api/large_response_json rules: rules: - - if: $API_FUZZING_DISABLED + - if: $API_FUZZING_DISABLED == 'true' || $API_FUZZING_DISABLED == '1' when: never - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME @@ -2414,7 +2423,7 @@ apifuzzer_branch: apifuzzer_main: extends: apifuzzer_fuzz rules: - - if: $API_FUZZING_DISABLED + - if: $API_FUZZING_DISABLED == 'true' || $API_FUZZING_DISABLED == '1' when: never - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME @@ -2697,14 +2706,75 @@ You can expedite the investigation with the [testssl.sh tool](https://testssl.sh 1. Run `./testssl.sh --log https://specs`. 1. Attach the log file to your support ticket. +### `ERROR: Job failed: failed to pull image` + +This error message occurs when pulling an image from a container registry that requires authentication to access (it is not public). + +In the job console output the error looks like: + +```log +Running with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a) + on blue-2.shared.runners-manager.gitlab.com/default XxUrkriX +Resolving secrets +00:00 +Preparing the "docker+machine" executor +00:06 +Using Docker executor with image registry.gitlab.com/security-products/api-security:2 ... +Starting service registry.example.com/my-target-app:latest ... +Pulling docker image registry.example.com/my-target-app:latest ... +WARNING: Failed to pull image with policy "always": Error response from daemon: Get https://registry.example.com/my-target-app/manifests/latest: unauthorized (manager.go:237:0s) +ERROR: Job failed: failed to pull image "registry.example.com/my-target-app:latest" with specified policies [always]: Error response from daemon: Get https://registry.example.com/my-target-app/manifests/latest: unauthorized (manager.go:237:0s) +``` + +**Error message** + +- In GitLab 15.9 and earlier, `ERROR: Job failed: failed to pull image` followed by `Error response from daemon: Get IMAGE: unauthorized`. + +**Solution** + +Authentication credentials are provided using the methods outlined in the [Access an image from a private Container Registry](../../../ci/docker/using_docker_images.md#access-an-image-from-a-private-container-registry) documentation section. The method used is dictated by your container registry provider and its configuration. If your using a container registry provided by a third party, such as a cloud provider (Azure, Google Could (GCP), AWS and so on), check the providers documentation for information on how to authenticate to their container registries. + +The following example uses the [statically defined credentials](../../../ci/docker/using_docker_images.md#use-statically-defined-credentials) authentication method. In this example the container registry is `registry.example.com` and image is `my-target-app:latest`. + +1. Read how to [Determine your `DOCKER_AUTH_CONFIG` data](../../../ci/docker/using_docker_images.md#determine-your-docker_auth_config-data) to understand how to compute the variable value for `DOCKER_AUTH_CONFIG`. The configuration variable `DOCKER_AUTH_CONFIG` contains the Docker JSON configuration to provide the appropriate authentication information. For example, to access private container registry: `registry.example.com` with the credentials `aGVsbG8gd29ybGQK`, the Docker JSON looks like: + + ```json + { + "auths": { + "registry.example.com": { + "auth": "aGVsbG8gd29ybGQK" + } + } + } + ``` + +1. Add the `DOCKER_AUTH_CONFIG` as a CI/CD variable. Instead of adding the configuration variable directly in your `.gitlab-ci.yml` file you should create a project [CI/CD variable](../../../ci/variables/index.md#for-a-project). +1. Rerun your job, and the statically-defined credentials are now used to sign in to the private container registry `registry.example.com`, and let you pull the image `my-target-app:latest`. If succeeded the job console shows an output like: + + ```log + Running with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a) + on blue-4.shared.runners-manager.gitlab.com/default J2nyww-s + Resolving secrets + 00:00 + Preparing the "docker+machine" executor + 00:56 + Using Docker executor with image registry.gitlab.com/security-products/api-security:2 ... + Starting service registry.example.com/my-target-app:latest ... + Authenticating with credentials from $DOCKER_AUTH_CONFIG + Pulling docker image registry.example.com/my-target-app:latest ... + Using docker image sha256:139c39668e5e4417f7d0eb0eeb74145ba862f4f3c24f7c6594ecb2f82dc4ad06 for registry.example.com/my-target-app:latest with digest registry.example.com/my-target- + app@sha256:2b69fc7c3627dbd0ebaa17674c264fcd2f2ba21ed9552a472acf8b065d39039c ... + Waiting for services to be up and running (timeout 30 seconds)... + ``` + ## Get support or request an improvement To get support for your particular problem use the [getting help channels](https://about.gitlab.com/get-help/). The [GitLab issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues) is the right place for bugs and feature proposals about API Security and API Fuzzing. -Use `~"Category:API Security"` [label](../../../development/contributing/issue_workflow.md#labels) when opening a new issue regarding API fuzzing to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response. +Use `~"Category:API Security"` [label](../../../development/labels/index.md) when opening a new issue regarding API fuzzing to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response. -[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an award emoji and or join the discussion. +[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an emoji reaction or join the discussion. When experiencing a behavior not working as expected, consider providing contextual information: diff --git a/doc/user/application_security/api_security/api_discovery/index.md b/doc/user/application_security/api_security/api_discovery/index.md new file mode 100644 index 00000000000..95b53249653 --- /dev/null +++ b/doc/user/application_security/api_security/api_discovery/index.md @@ -0,0 +1,181 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference, howto +--- + +# API Discovery **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/9302) in GitLab 15.9. The API Discovery feature is in [Beta](../../../../policy/alpha-beta-support.md). + +API Discovery analyzes your application and produces an OpenAPI document describing the web APIs it exposes. This schema document can then be used by [DAST API](../../dast_api/index.md) or [API Fuzzing](../../api_fuzzing/index.md) to perform security scans of the web API. + +## Supported frameworks + +- [Java Spring-Boot](#java-spring-boot) + +## When does API Discovery run? + +API Discovery runs as a standalone job in your pipeline. The resulting OpenAPI document is captured as a job artifact so it can be used by other jobs in later stages. + +API Discovery runs in the `test` stage by default. The `test` stage was chosen as it typically executes before the stages used by other API Security features such as DAST API and API Fuzzing. + +## Example API Discovery configurations + +The following projects demonstrate API Discovery: + +- [Example Java Spring Boot v2 Pet Store](https://gitlab.com/gitlab-org/security-products/demos/api-discovery/java-spring-boot-v2-petstore) + +## Java Spring-Boot + +[Spring Boot](https://spring.io/projects/spring-boot) is a popular framework for creating stand-alone, production-grade Spring-based applications. + +### Supported Applications + +- Spring Boot: v2.X (>= 2.1) +- Java: 11, 17 (LTS versions) +- Executable JARs + +API Discovery supports Spring Boot major version 2, minor versions 1 and later. Versions 2.0.X are not supported due to known bugs which affect API Discovery and were fixed in 2.1. + +Major version 3 is planned to be supported in the future. Support for major version 1 is not planned. + +API Discovery is tested with and officially supports LTS versions of the Java runtime. Other versions may work also, and bug reports from non-LTS versions are welcome. + +Only applications that are built as Spring Boot [executable JARs](https://docs.spring.io/spring-boot/docs/current/reference/html/executable-jar.html#appendix.executable-jar.nested-jars.jar-structure) are supported. + +### Configure as pipeline job + +The easiest way to run API Discovery is through a pipeline job based on our CI template. +When running in this method, you provide a container image that has the required dependencies installed (such as an appropriate Java runtime). See [Image Requirements](#image-requirements) for more information. + +1. A container image that meets the [image requirements](#image-requirements) is uploaded to a container registry. If the container registry requires authentication see [this help section](/ee/ci/docker/using_docker_images.md#access-an-image-from-a-private-container-registry). +1. In a job in the `build` stage, build your application and configure the resulting Spring Boot executable JAR as a job artifact. +1. Include the API Discovery template in your `.gitlab-ci.yml` file. + + ```yaml + include: + - template: Security/API-Discovery.gitlab-ci.yml + ``` + + Only a single `include` statement is allowed per `.gitlab-ci.yml` file. If you are including other files, combine them into a single `include` statement. + + ```yaml + include: + - template: Security/API-Discovery.gitlab-ci.yml + - template: Security/DAST-API.gitlab-ci.yml + ``` + +1. Create a new job that extends from `.api_discovery_java_spring_boot`. The default stage is `test` which can be optionally changed to any value. + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + ``` + +1. Configure the `image` for the job. + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + image: openjdk:11-jre-slim + ``` + +1. Provide the Java class path needed by your application. This includes your compatible build + artifact from step 2, along with any additional dependencies. For this example, the build artifact + is `build/libs/spring-boot-app-0.0.0.jar` and contains all needed dependencies. The variable + `API_DISCOVERY_JAVA_CLASSPATH` is used to provide the class path. + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + image: openjdk:11-jre-slim + variables: + API_DISCOVERY_JAVA_CLASSPATH: build/libs/spring-boot-app-0.0.0.jar + ``` + +1. Optional. If the image provided is missing a dependency needed by API Discovery, it can be added + using a `before_script`. In this example, the `openjdk:11-jre-slim` container doesn't include + `curl` which is required by API Discovery. The dependency can be installed using the Debian + package manager `apt`: + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + image: openjdk:11-jre-slim + variables: + API_DISCOVERY_JAVA_CLASSPATH: build/libs/spring-boot-app-0.0.0.jar + before_script: + - apt-get update && apt-get install -y curl + ``` + +1. Optional. If the image provided doesn't automatically set the `JAVA_HOME` environment variable, + or include `java` in the path, the `API_DISCOVERY_JAVA_HOME` variable can be used. + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + image: openjdk:11-jre-slim + variables: + API_DISCOVERY_JAVA_CLASSPATH: build/libs/spring-boot-app-0.0.0.jar + API_DISCOVERY_JAVA_HOME: /opt/java + ``` + +1. Optional. If the package registry at `API_DISCOVERY_PACKAGES` is not public, provide a token that + has read access to the GitLab API and registry using the `API_DISCOVERY_PACKAGE_TOKEN` variable. + This is not required if you are using `gitlab.com` and have not customized the `API_DISCOVERY_PACKAGES` + variable. The following example uses a + [custom CI/CD variable](../../../../ci/variables/index.md#define-a-cicd-variable-in-the-ui) named + `GITLAB_READ_TOKEN` to store the token. + + ```yaml + api_discovery: + extends: .api_discovery_java_spring_boot + image: openjdk:8-jre-alpine + variables: + API_DISCOVERY_JAVA_CLASSPATH: build/libs/spring-boot-app-0.0.0.jar + API_DISCOVERY_PACKAGE_TOKEN: $GITLAB_READ_TOKEN + ``` + +After the API Discovery job has successfully run, the OpenAPI document is available as a job artifact called `gl-api-discovery-openapi.json`. + +#### Image requirements + +- Linux container image. +- Java versions 11 or 17 are officially supported, but other versions are likely compatible as well. +- The `curl` command. +- A shell at `/bin/sh` (like `busybox`, `sh`, or `bash`). + +### Available CI/CD variables + +| CI/CD variable | Description | +|---------------------------------------------|--------------------| +| `API_DISCOVERY_DISABLED` | Disables the API Discovery job when using template job rules. | +| `API_DISCOVERY_DISABLED_FOR_DEFAULT_BRANCH` | Disables the API Discovery job for default branch pipelines when using template job rules. | +| `API_DISCOVERY_JAVA_CLASSPATH` | Java class-path that includes target Spring Boot application. (`build/libs/sample-0.0.0.jar`) | +| `API_DISCOVERY_JAVA_HOME` | If provided is used to set `JAVA_HOME`. | +| `API_DISCOVERY_PACKAGES` | GitLab Project Package API Prefix (defaults to `$CI_API_V4_URL/projects/42503323/packages`). | +| `API_DISCOVERY_PACKAGE_TOKEN` | GitLab token for calling the GitLab package API. Only needed when `API_DISCOVERY_PACKAGES` is set to a non-public project. | +| `API_DISCOVERY_VERSION` | API Discovery version to use (defaults to `1`). Can be used to pin a version by providing the full version number `1.1.0`. | + +## Get support or request an improvement + +To get support for your particular problem, use the [getting help channels](https://about.gitlab.com/get-help/). + +The [GitLab issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues) is the right place for bugs and feature proposals about API Discovery. +Use `~"Category:API Security"` [label](../../../../development/labels/index.md) when opening a new issue regarding API Discovery to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response. + +[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an emoji reaction or join the discussion. + +When experiencing a behavior not working as expected, consider providing contextual information: + +- GitLab version if using a self-managed instance. +- `.gitlab-ci.yml` job definition. +- Full job console output. +- Framework in use with version (for example Spring Boot v2.3.2). +- Language runtime with version (for example OpenJDK v17.0.1). +<!-- - Scanner log file is available as a job artifact named `gl-api-discovery.log`. --> + +WARNING: +**Sanitize data attached to a support issue**. Remove sensitive information, including: credentials, passwords, tokens, keys, and secrets. diff --git a/doc/user/application_security/api_security/index.md b/doc/user/application_security/api_security/index.md new file mode 100644 index 00000000000..5c2e74bceae --- /dev/null +++ b/doc/user/application_security/api_security/index.md @@ -0,0 +1,21 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference, howto +--- + +# API Security **(ULTIMATE)** + +API Security refers to the measures taken to secure and protect web Application Programming Interfaces (APIs) from unauthorized access, misuse, and attacks. +APIs are a crucial component of modern application development as they allow applications to interact with each other and exchange data. +However, this also makes them attractive to attackers and vulnerable to security threats if not properly secured. +In this section, we discuss GitLab features that can be used to ensure the security of web APIs in your application. +Some of the features discussed are specific to web APIs and others are more general solutions that are also used with web API applications. + +- [SAST](../sast) identified vulnerabilities by analyzing the application's codebase. +- [Dependency Scanning](../dependency_scanning) reviews a project 3rd party dependencies for known vulnerabilities (for example CVEs). +- [Container Scanning](../container_scanning) analyzes container images to identify known OS package vulnerabilities and installed language dependencies. +- [API Discovery](api_discovery) examines an application containing a REST API and intuits an OpenAPI specification for that API. OpenAPI specification documents are used by other GitLab security tools. +- [DAST API](../dast_api) performs dynamic analysis security testing of web APIs. It can identify various security vulnerabilities in your application, including the OWASP Top 10. +- [API Fuzzing](../api_fuzzing) performs fuzz testing of a web API. Fuzz testing looks for issues in an application that are not previously known and don't map to classic vulnerability types such as SQL Injection. diff --git a/doc/user/application_security/breach_and_attack_simulation/index.md b/doc/user/application_security/breach_and_attack_simulation/index.md new file mode 100644 index 00000000000..bb67150d4fa --- /dev/null +++ b/doc/user/application_security/breach_and_attack_simulation/index.md @@ -0,0 +1,141 @@ +--- +stage: Secure +group: Incubation +info: Breach and Attack Simulation is a GitLab Incubation Engineering program. No technical writer assigned to this group. +type: reference, howto +--- + +# Breach and Attack Simulation **(ULTIMATE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/402784) in GitLab 15.11 as an Incubating feature. +> - [Included](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119981) in the `Security/BAS.latest.gitlab-ci.yml` in GitLab 16.0. + +DISCLAIMER: +Breach and Attack Simulation is a set of incubating features being developed by the Incubation Engineering Department and is subject to significant changes over time. + +Breach and Attack Simulation (BAS) uses additional security testing techniques to assess the risk of detected vulnerabilities and prioritize the remediation of exploitable vulnerabilities. + +For feedback, bug reports, and feature requests, see the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/404809). + +WARNING: +Only run BAS scans against test servers. Testing attacker behavior can lead to modification or loss of data. + +## Extend Dynamic Application Security Testing (DAST) + +You can simulate attacks with [DAST](../dast/index.md) to detect vulnerabilities. +By default, DAST active checks match an expected response, or determine by response +time whether a vulnerability was exploited. + +To enable BAS extended DAST scanning for your application, use the `dast_with_bas` job defined +in the GitLab BAS CI/CD template file. Updates to the template are provided with GitLab +upgrades, allowing you to benefit from any improvements and additions. + +1. Include the appropriate CI/CD template: + + - [`BAS.latest.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/BAS.latest.gitlab-ci.yml): + Latest version of the BAS template. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119981) + in GitLab 16.0). + + WARNING: + The latest version of the template may include breaking changes. Use the + stable template unless you need a feature provided only in the latest template. + + For more information about template versioning, see the [CI/CD documentation](../../../development/cicd/templates.md#latest-version). + +1. Choose one of the following options for running BAS extended DAST scans: + + - [Enable a separate BAS extended DAST job](#enable-a-separate-bas-extended-dast-job) + + - You're not using the latest DAST template yet. + - Continue using a stable version of the DAST security analyzer image for DAST scans. + - Create a duplicate `dast_with_bas` job which extends your existing DAST job configuration. + + - [Extend an existing DAST job](#extend-an-existing-dast-job) + - You're already using the latest DAST template rather than the stable template. + - Extend your existing DAST job to include the latest DAST security analyzer image tag from the Breach and Attack Simulation SEG. + +1. Setup a callback server to [enable callback attacks](#enable-callback-attacks). + +### Enable a separate BAS extended DAST job + +To maintain a separate DAST job while testing the BAS extended DAST image: + +1. Add a `dast` stage to your GitLab CI/CD stages configuration. + + ```yaml + stages: + - build + - test + - deploy + - dast + ``` + +1. Set the `DAST_WEBSITE` [CI/CD variable](../../../ci/yaml/index.md#variables). + + ```yaml + dast_with_bas: + variables: + DAST_WEBSITE: http://yourapp + ``` + +### Extend an existing DAST job + +To enable Breach and Attack Simulation features inside of an existing DAST job: + +1. Follow the steps in [Create a DAST CI/CD job](../dast/browser_based.md#create-a-dast-cicd-job). + +1. Extend DAST to using the [extends](../../../ci/yaml/yaml_optimization.md#use-extends-to-reuse-configuration-sections) keyword to your DAST job's configuration: + + ```yaml + dast: + extends: .dast_with_bas + ``` + +1. Disable the `dast+job` job included in the BAS template by setting `DAST_BAS_DISABLED`: + + ```yaml + variables: + DAST_BAS_DISABLED: "true" + ``` + +### Enable callback attacks + +DISCLAIMER: +This page contains information related to upcoming products, features, and functionality. +It is important to note that the information presented is for informational purposes only. +Please do not rely on this information for purchasing or planning purposes. +As with all projects, the items mentioned on this page are subject to change or delay. +The development, release, and timing of any products, features, or functionality remain at the +sole discretion of GitLab Inc. + +Perform Out-of-Band Application Security Testing (OAST) for certain [active checks](../dast/checks/index.md#active-checks). + +1. Extend the `.dast_with_bas_using_services` job configuration using the [extends](../../../ci/yaml/yaml_optimization.md#use-extends-to-reuse-configuration-sections) keyword: + + ```yaml + dast: + extends: .dast_with_bas_using_services + + dast_with_bas: + extends: + # NOTE: extends overwrites rather than merges so dast must be included in this list. + - dast + - .dast_with_bas_using_services + ``` + +1. Use a [!reference tag](../../../ci/yaml/yaml_optimization.md#reference-tags) to pull in the default `callback` service container in your `services`. + + ```yaml + services: + # NOTE: services overwrites rather than merges so it must be referenced to merge. + - !reference [.dast_with_bas_using_services, services] + - name: $CI_REGISTRY_IMAGE + alias: yourapp + ``` + +You can also manually enable callback attacks by making sure to: + +1. Set the `DAST_FF_ENABLE_BAS` [CI/CD variable](../dast/browser_based.md#available-cicd-variables) to `true`. +1. Enable both the application being tested and callback service container using [services](../../../ci/services/index.md). +1. Enable container-to-container networking [making the callback service accessible](../../../ci/services/index.md#connecting-services) in the job. +1. Set `DAST_BROWSER_CALLBACK` to include `Address:$YOUR_CALLBACK_URL` key/value pair where the callback service is accessible to the Runner/DAST container. diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index ea422f0b33c..d5a05477ddf 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -5,7 +5,7 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Security Configuration **(FREE)** +# Security configuration **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in GitLab 12.6. > - SAST configuration was [enabled](https://gitlab.com/groups/gitlab-org/-/epics/3659) in 13.3 and [improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in 13.4. @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - A simplified version was made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/294076) in GitLab 13.10. > - [Redesigned](https://gitlab.com/gitlab-org/gitlab/-/issues/326926) in 14.2. -The Security Configuration page lists the following for the security testing and compliance tools: +The **Security configuration** page lists the following for the security testing and compliance tools: - Name, description, and a documentation link. - Whether or not it is available. @@ -41,7 +41,7 @@ all security features are configured by default. To view a project's security configuration: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. Select **Configuration history** to see the `.gitlab-ci.yml` file's history. @@ -51,7 +51,7 @@ You can configure the following security controls: - [Static Application Security Testing](../sast/index.md) (SAST) - Select **Enable SAST** to configure SAST for the current project. - For more details, read [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). + For more details, read [Configure SAST in the UI](../sast/index.md#configure-sast-by-using-the-ui). - [Dynamic Application Security Testing](../dast/index.md) (DAST) - Select **Enable DAST** to configure DAST for the current project. - Select **Manage scans** to manage the saved DAST scans, site profiles, and scanner profiles. diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 0a586a14cc4..7a82f98425a 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -15,12 +15,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86092) the major analyzer version from `4` to `5` in GitLab 15.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86783) from GitLab Ultimate to GitLab Free in 15.0. > - Container Scanning variables that reference Docker [renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/357264) in GitLab 15.4. +> - Container Scanning template [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/381665) from `Security/Container-Scanning.gitlab-ci.yml` to `Jobs/Container-Scanning.gitlab-ci.yml` in GitLab 15.6. Your application's Docker image may itself be based on Docker images that contain known vulnerabilities. By including an extra Container Scanning job in your pipeline that scans for those vulnerabilities and displays them in a merge request, you can use GitLab to audit your Docker-based apps. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Container Scanning](https://www.youtube.com/watch?v=C0jn2eN5MAs). + Container Scanning is often considered part of Software Composition Analysis (SCA). SCA can contain aspects of inspecting the items your code uses. These items typically include application and system dependencies that are almost always imported from external sources, rather than sourced from items @@ -71,14 +75,14 @@ information directly in the merge request. | [Access to Security Dashboard page](#security-dashboard) | No | Yes | | [Access to Dependency List page](../dependency_list/index.md) | No | Yes | -## Requirements +## Prerequisites To enable container scanning in your pipeline, you need the following: - GitLab CI/CD pipeline must include the `test` stage, which is available unless overridden with the [`stages`](../../../ci/yaml/index.md#stages) keyword. - [GitLab Runner](https://docs.gitlab.com/runner/) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor on Linux/amd64. -- Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the +- Docker `18.09.03` or later installed on the same computer as the runner. If you're using the shared runners on GitLab.com, then this is already the case. - An image matching the [supported distributions](#supported-distributions). - [Build and push](../../packages/container_registry/build_and_push_images.md#use-gitlab-cicd) @@ -90,19 +94,19 @@ To enable container scanning in your pipeline, you need the following: ## Configuration To enable container scanning, add the -[`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml) +[`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) to your `.gitlab-ci.yml` file: ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml ``` The included template: - Creates a `container_scanning` job in your CI/CD pipeline. - Pulls the built Docker image from your project's [container registry](../../packages/container_registry/index.md) - (see [requirements](#requirements)) and scans it for possible vulnerabilities. + (see [prerequisites](#prerequisites)) and scans it for possible vulnerabilities. GitLab saves the results as a [Container Scanning report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportscontainer_scanning) @@ -117,7 +121,7 @@ registry, and scans the image: ```yaml include: - template: Jobs/Build.gitlab-ci.yml - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -142,7 +146,7 @@ enables verbose output for the analyzer: ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml variables: SECURE_LOG_LEVEL: 'debug' @@ -154,7 +158,7 @@ To scan images located in a registry other than the project's, use the following ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -178,7 +182,9 @@ container_scanning: - export AWS_ECR_PASSWORD=$(aws ecr get-login-password --region region) include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml + +variables: CS_IMAGE: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<image>:<tag> CS_REGISTRY_USER: AWS CS_REGISTRY_PASSWORD: "$AWS_ECR_PASSWORD" @@ -199,7 +205,7 @@ For example: ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -223,7 +229,7 @@ By default, the report only includes packages managed by the Operating System (O ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -251,7 +257,7 @@ including a large number of false positives. | `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | All | | `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | All | | `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | All | -| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:5` | Docker image of the analyzer. | All | +| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:6` | Docker image of the analyzer. | All | | `CS_DEFAULT_BRANCH_IMAGE` | `""` | The name of the `CS_IMAGE` on the default branch. See [Setting the default branch image](#setting-the-default-branch-image) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5. | All | | `CS_DISABLE_DEPENDENCY_LIST` | `"false"` | Disable Dependency Scanning for packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` | `"true"` | Disable scanning for language-specific packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | @@ -260,10 +266,6 @@ including a large number of false positives. | `CS_IGNORE_UNFIXED` | `"false"` | Ignore vulnerabilities that are not fixed. | All | | `CS_REGISTRY_INSECURE` | `"false"` | Allow access to insecure registries (HTTP only). Should only be set to `true` when testing the image locally. Works with all scanners, but the registry must listen on port `80/tcp` for Trivy to work. | All | | `CS_SEVERITY_THRESHOLD` | `UNKNOWN` | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are `UNKNOWN`, `LOW`, `MEDIUM`, `HIGH`, and `CRITICAL`. | Trivy | -| <!-- start_remove The following content will be removed on remove_date: '2023-08-22' --> `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_IMAGE`. The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | -| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_PASSWORD`. Password for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | -| `DOCKER_USER` | `$CI_REGISTRY_USER` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_USER`. Username for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | -| `DOCKERFILE_PATH` | `Dockerfile` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_DOCKERFILE_PATH`. The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | All <!-- end_remove --> | | `CS_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | | `CS_REGISTRY_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. The default is only set if `$CS_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | | `CS_REGISTRY_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. The default is only set if `$CS_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All | @@ -303,9 +305,9 @@ standard tag plus the `-fips` extension. | Scanner name | `CS_ANALYZER_IMAGE` | | --------------- | ------------------- | -| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:5-fips` | -| Grype | `registry.gitlab.com/security-products/container-scanning/grype:5-fips` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5-fips` | +| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:6-fips` | +| Grype | `registry.gitlab.com/security-products/container-scanning/grype:6-fips` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:6-fips` | NOTE: Prior to GitLab 15.0, the `-ubi` image extension is also available. GitLab 15.0 and later only @@ -326,7 +328,7 @@ To enable Container Scanning in a project, create a merge request from the Secur page: 1. In the project where you want to enable Container Scanning, go to - **Security & Compliance > Configuration**. + **Security and Compliance > Security configuration**. 1. In the **Container Scanning** row, select **Configure with a merge request**. This automatically creates a merge request with the changes necessary to enable Container Scanning. @@ -346,7 +348,7 @@ This example sets `GIT_STRATEGY` to `fetch`: ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -362,9 +364,9 @@ The following options are available: | Scanner name | `CS_ANALYZER_IMAGE` | |----------------------------------------------------------|--------------------------------------------------------------------| -| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:5` | -| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:5` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5` | +| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:6` | +| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:6` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:6` | ### Setting the default branch image @@ -392,7 +394,7 @@ duplicated: ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -525,7 +527,7 @@ successfully run. For more information, see [Offline environments](../offline_de To use container scanning in an offline environment, you need: -- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- GitLab Runner with the [`docker` or `kubernetes` executor](#prerequisites). - To configure a local Docker container registry with copies of the container scanning images. You can find these images in their respective registries: @@ -555,9 +557,9 @@ For container scanning, import the following images from `registry.gitlab.com` i [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/container-scanning:5 -registry.gitlab.com/security-products/container-scanning/grype:5 -registry.gitlab.com/security-products/container-scanning/trivy:5 +registry.gitlab.com/security-products/container-scanning:6 +registry.gitlab.com/security-products/container-scanning/grype:6 +registry.gitlab.com/security-products/container-scanning/trivy:6 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -578,7 +580,7 @@ For details on saving and transporting Docker images as a file, see the Docker d ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: image: $CI_REGISTRY/namespace/container-scanning @@ -597,7 +599,7 @@ following `.gitlab-ci.yml` example as a template. ```yaml variables: - SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:5 + SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:6 TARGET_IMAGE: $CI_REGISTRY/namespace/container-scanning image: docker:stable @@ -629,7 +631,7 @@ This example shows the configuration needed to scan images in a private [Google ```yaml include: - - template: Jobs/Container-Scanning.gitlab-ci.yml + - template: Security/Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -693,6 +695,14 @@ These reports must follow a format defined in the For more information, see [Security scanner integration](../../../development/integrations/secure.md). +### CycloneDX Software Bill of Materials + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396381) in GitLab 15.11. + +In addition to the [JSON report file](#reports-json-format), the [Container Scanning](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning) tool outputs a [CycloneDX](https://cyclonedx.org/) Software Bill of Materials (SBOM) for the scanned image. This CycloneDX SBOM is named `gl-sbom-report.cdx.json` and is saved in the same directory as the `JSON report file`. This feature is only supported when the `Trivy` analyzer is used. + +You can download CycloneDX SBOMs [the same way as other job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). + ## Security Dashboard The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all @@ -771,7 +781,7 @@ To prevent the error, ensure the Docker version that the runner is using is ### Getting warning message `gl-container-scanning-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ## Changes diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 5d2593a1bed..09370a9a7f5 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -15,6 +15,9 @@ We recommend that you use fuzz testing in addition to the other security scanner and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run your coverage-guided fuzz testing as part your CI/CD workflow. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Coverage Fuzzing](https://www.youtube.com/watch?v=bbIenVVcjW0). + ## Coverage-guided fuzz testing process The fuzz testing process: @@ -40,7 +43,7 @@ You can use the following fuzzing engines to test the specified languages. | Language | Fuzzing Engine | Example | |---------------------------------------------|------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------| | C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | [c-cpp-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/c-cpp-fuzzing-example) | -| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example) | +| Go | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example) | | Swift | [libFuzzer](https://github.com/apple/swift/blob/master/docs/libFuzzerIntegration.md) | [swift-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/swift-fuzzing-example) | | Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | [rust-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/rust-fuzzing-example) | | Java | [Javafuzz](https://gitlab.com/gitlab-org/security-products/analyzers/fuzzers/javafuzz) (recommended) | [javafuzz-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/javafuzz-fuzzing-example) | @@ -54,7 +57,7 @@ You can use the following fuzzing engines to test the specified languages. To confirm the status of coverage-guided fuzz testing: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Coverage Fuzzing** section the status is: - **Not configured** - **Enabled** @@ -145,7 +148,7 @@ Each fuzzing step outputs these artifacts: previous jobs. You can download the JSON report file from the CI/CD pipelines page. For more information, see -[Downloading artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[Downloading artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). ## Corpus registry @@ -168,7 +171,7 @@ artifacts files you can download from the CI/CD pipeline. Also, a project member To view details of the corpus registry: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Coverage Fuzzing** section, select **Manage corpus**. ### Create a corpus in the corpus registry @@ -196,7 +199,7 @@ provided by the `COVFUZZ_CORPUS_NAME` variable. The corpus is updated on every p To upload an existing corpus file: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Coverage Fuzzing** section, select **Manage corpus**. 1. Select **New corpus**. 1. Complete the fields. @@ -222,7 +225,7 @@ Prerequisites: ## Coverage-guided fuzz testing report -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](../../../policy/alpha-beta-support.md#alpha-features). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Experiment](../../../policy/alpha-beta-support.md#experiment). For detailed information about the `gl-coverage-fuzzing-report.json` file's format, read the [schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json). diff --git a/doc/user/application_security/dast/authentication.md b/doc/user/application_security/dast/authentication.md index 77732ab532c..1a68abd01f6 100644 --- a/doc/user/application_security/dast/authentication.md +++ b/doc/user/application_security/dast/authentication.md @@ -32,7 +32,7 @@ Authentication supports single-step login forms, multi-step login forms, single ## Getting started NOTE: -We recommend periodically confirming that the analyzer's authentication is still working, as this tends to break over +You should periodically confirming that the analyzer's authentication is still working, as this tends to break over time due to changes to the application. To run a DAST authenticated scan: @@ -45,34 +45,45 @@ To run a DAST authenticated scan: ### Prerequisites -- You are using either the [DAST proxy-based analyzer](proxy-based.md) or the [DAST browser-based analyzer](browser_based.md). -- You know the URL of the login form of your application. Alternatively, you know how to navigate to the login form from the authentication URL (see [clicking to navigate to the login form](#clicking-to-navigate-to-the-login-form)). - You have the username and password of the user you would like to authenticate as during the scan. +- You have checked the [known limitations](#known-limitations) to ensure DAST can authenticate to your application. +- You have satisfied the prerequisites depending on whether you're using [form authentication](#form-authentication) or [HTTP authentication](#http-authentication). +- You have thought about how you can [verify](#verifying-authentication-is-successful) whether or not authentication was successful. + +#### Form authentication + +- You are using either the [DAST proxy-based analyzer](proxy-based.md) or the [DAST browser-based analyzer](browser_based.md). +- You know the URL of the login form of your application. Alternatively, you know how to go to the login form from the authentication URL (see [clicking to go to the login form](#clicking-to-go-to-the-login-form)). - You know the [selectors](#finding-an-elements-selector) of the username and password HTML fields that DAST uses to input the respective values. - You know the element's [selector](#finding-an-elements-selector) that submits the login form when selected. -- You have thought about how you can [verify](#verifying-authentication-is-successful) whether or not authentication was successful. -- You have checked the [known limitations](#known-limitations) to ensure DAST can authenticate to your application. + +#### HTTP authentication + +- You must be using the [DAST browser-based analyzer](browser_based.md). ### Available CI/CD variables | CI/CD variable | Type | Description | |:-----------------------------------------------|:------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `DAST_AUTH_COOKIES` | string | Set to a comma-separated list of cookie names to specify which cookies are used for authentication. | -| `DAST_AUTH_REPORT` | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | +| `DAST_AUTH_REPORT` | boolean | Set to `true` to generate a report detailing steps taken during the authentication process. You must also define `gl-dast-debug-auth-report.html` as a CI job artifact to be able to access the generated report. The report's content aids when debugging authentication failures. | +| `DAST_AUTH_TYPE` <sup>2</sup> | string | The authentication type to use. Example: `basic-digest`. | | `DAST_AUTH_URL` <sup>1</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Example: `https://login.example.com`. | -| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the absence of a login form once the login form has been submitted. | -| `DAST_AUTH_VERIFICATION_SELECTOR` | [selector](#finding-an-elements-selector) | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo`. | -| `DAST_AUTH_VERIFICATION_URL` <sup>1</sup> | URL | Verifies successful authentication by checking the URL in the browser once the login form has been submitted. Example: `"https://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | +| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the absence of a login form after the login form has been submitted. | +| `DAST_AUTH_VERIFICATION_SELECTOR` | [selector](#finding-an-elements-selector) | Verifies successful authentication by checking for presence of a selector after the login form has been submitted. Example: `css:.user-photo`. | +| `DAST_AUTH_VERIFICATION_URL` <sup>1</sup> | URL | Verifies successful authentication by checking the URL in the browser after the login form has been submitted. Example: `"https://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | | `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1</sup> | [selector](#finding-an-elements-selector) | Comma-separated list of selectors that are selected prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | | `DAST_EXCLUDE_URLS` <sup>1</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | -| `DAST_FIRST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | +| `DAST_FIRST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | | `DAST_PASSWORD` <sup>1</sup> | string | The password to authenticate to in the website. Example: `P@55w0rd!` | | `DAST_PASSWORD_FIELD` | string | The selector of password field at the sign-in HTML form. Example: `id:password` | -| `DAST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | +| `DAST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9894) in GitLab 12.4. | | `DAST_USERNAME` <sup>1</sup> | string | The username to authenticate to in the website. Example: `admin` | | `DAST_USERNAME_FIELD` <sup>1</sup> | string | The selector of username field at the sign-in HTML form. Example: `name:username` | +| `DAST_AUTH_DISABLE_CLEAR_FIELDS` | boolean | Disables clearing of username and password fields before attempting manual login. Set to `false` by default. | 1. Available to an on-demand proxy-based DAST scan. +1. Not available to proxy-based scans. ### Update the target website @@ -93,12 +104,34 @@ dast: DAST_AUTH_URL: "https://example.com/login" ``` +### Configuration for HTTP authentication + +To use an [HTTP authentication scheme](https://www.chromium.org/developers/design-documents/http-authentication/) such as Basic Authentication you can set the `DAST_AUTH_TYPE` value to `basic-digest`. +Other schemes such as Negotiate or NTLM may work but aren't officially supported due to current lack of automated test coverage. + +Configuration requires the CI/CD variables `DAST_AUTH_TYPE`, `DAST_AUTH_URL`, `DAST_USERNAME`, `DAST_PASSWORD` to be defined for the DAST job. If you don't have a unique login URL, set `DAST_AUTH_URL` to the same URL as `DAST_WEBSITE`. + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://example.com" + DAST_AUTH_TYPE: "basic-digest" + DAST_AUTH_URL: "https://example.com" +``` + +Do **not** define `DAST_USERNAME` and `DAST_PASSWORD` in the YAML job definition file as this could present a security risk. Instead, create them as masked CI/CD variables using the GitLab UI. +See [Custom CI/CD variables](../../../ci/variables/index.md#for-a-project) for more information. +The proxy-based analyzer does not support basic authentication as an authentication mechanism. A workaround could be to set `DAST_REQUEST_HEADERS` as a masked CI/CD variable with a value containing the appropriate `Authorization` header, for example, `Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK`. + ### Configuration for a single-step login form A single-step login form has all login form elements on a single page. Configuration requires the CI/CD variables `DAST_AUTH_URL`, `DAST_USERNAME`, `DAST_USERNAME_FIELD`, `DAST_PASSWORD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD` to be defined for the DAST job. -It is recommended to set up the URL and selectors of fields in the job definition YAML, for example: +You should set up the URL and selectors of fields in the job definition YAML, for example: ```yaml include: @@ -114,16 +147,24 @@ dast: ``` Do **not** define `DAST_USERNAME` and `DAST_PASSWORD` in the YAML job definition file as this could present a security risk. Instead, create them as masked CI/CD variables using the GitLab UI. -See [Custom CI/CI variables](../../../ci/variables/index.md#for-a-project) for more information. +See [Custom CI/CD variables](../../../ci/variables/index.md#for-a-project) for more information. ### Configuration for a multi-step login form A multi-step login form has two pages. The first page has a form with the username and a next submit button. If the username is valid, a second form on the subsequent page has the password and the form submit button. -Configuration requires the CI/CD variables `DAST_AUTH_URL`, `DAST_USERNAME`, `DAST_USERNAME_FIELD`, `DAST_FIRST_SUBMIT_FIELD`, `DAST_PASSWORD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD` to be defined for the DAST job. +Configuration requires the CI/CD variables to be defined for the DAST job: + +- `DAST_AUTH_URL` +- `DAST_USERNAME` +- `DAST_USERNAME_FIELD` +- `DAST_FIRST_SUBMIT_FIELD` +- `DAST_PASSWORD` +- `DAST_PASSWORD_FIELD` +- `DAST_SUBMIT_FIELD`. -It is recommended to set up the URL and selectors of fields in the job definition YAML, for example: +You should set up the URL and selectors of fields in the job definition YAML, for example: ```yaml include: @@ -140,21 +181,21 @@ dast: ``` Do **not** define `DAST_USERNAME` and `DAST_PASSWORD` in the YAML job definition file as this could present a security risk. Instead, create them as masked CI/CD variables using the GitLab UI. -See [Custom CI/CI variables](../../../ci/variables/index.md#for-a-project) for more information. +See [Custom CI/CD variables](../../../ci/variables/index.md#for-a-project) for more information. ### Configuration for Single Sign-On (SSO) If a user can log into an application, then in most cases, DAST is also able to log in. -This is the case even when an application uses Single Sign-on. Applications using SSO solutions should configure DAST +Even when an application uses Single Sign-on. Applications using SSO solutions should configure DAST authentication using the [single-step](#configuration-for-a-single-step-login-form) or [multi-step](#configuration-for-a-multi-step-login-form) login form configuration guides. DAST supports authentication processes where a user is redirected to an external Identity Provider's site to log in. Check the [known limitations](#known-limitations) of DAST authentication to determine if your SSO authentication process is supported. -### Clicking to navigate to the login form +### Clicking to go to the login form Define `DAST_BROWSER_PATH_TO_LOGIN_FORM` to provide a path of elements to click on from the `DAST_AUTH_URL` so that DAST can access the -login form. This is useful for applications that show the login form in a pop-up (modal) window or when the login form does not +login form. This method is suitable for applications that show the login form in a pop-up (modal) window or when the login form does not have a unique URL. For example: @@ -204,12 +245,12 @@ Selectors have the format `type`:`search string`. DAST searches for the selector Chrome DevTools element selector tool is an effective way to find a selector. -1. Open Chrome and navigate to the page where you would like to find a selector, for example, the login page for your site. +1. Open Chrome and go to the page where you would like to find a selector, for example, the login page for your site. 1. Open the `Elements` tab in Chrome DevTools with the keyboard shortcut `Command + Shift + c` in macOS or `Ctrl + Shift + c` in Windows. 1. Select the `Select an element in the page to select it` tool. ![search-elements](img/dast_auth_browser_scan_search_elements.png) 1. Select the field on your page that you would like to know the selector for. -1. Once the tool is active, highlight a field you wish to view the details of. +1. After the tool is active, highlight a field you wish to view the details of. ![highlight](img/dast_auth_browser_scan_highlight.png) 1. Once highlighted, you can see the element's details, including attributes that would make a good candidate for a selector. @@ -220,15 +261,15 @@ In this example, the `id="user_login"` appears to be a good candidate. You can u Judicious choice of selector leads to a scan that is resilient to the application changing. -In order of preference, it is recommended to choose as selectors: +In order of preference, you should choose as selectors: -- `id` fields. These are generally unique on a page, and rarely change. -- `name` fields. These are generally unique on a page, and rarely change. +- `id` fields. These fields generally unique on a page, and rarely change. +- `name` fields. These fields generally unique on a page, and rarely change. - `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field. - Presence of field specific data attributes, such as the selector, `"css:[data-username]"` when the `data-username` field has any value on the username field. - Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`. -When using selectors to locate specific fields we recommend you avoid searching on: +When using selectors to locate specific fields you should avoid searching on: - Any `id`, `name`, `attribute`, `class` or `value` that is dynamically generated. - Generic class names, such as `column-10` and `dark-grey`. @@ -237,7 +278,7 @@ When using selectors to locate specific fields we recommend you avoid searching ## Verifying authentication is successful -Once DAST has submitted the login form, a verification process takes place +After DAST has submitted the login form, a verification process takes place to determine if authentication succeeded. The scan halts with an error if authentication is unsuccessful. Following the submission of the login form, authentication is determined to be unsuccessful when: @@ -255,7 +296,7 @@ DAST tests for the absence of a login form if no verification checks are configu #### Verify based on the URL -Define `DAST_AUTH_VERIFICATION_URL` as the URL displayed in the browser tab once the login form is successfully submitted. +Define `DAST_AUTH_VERIFICATION_URL` as the URL displayed in the browser tab after the login form is successfully submitted. DAST compares the verification URL to the URL in the browser after authentication. If they are not the same, authentication is unsuccessful. @@ -275,7 +316,7 @@ dast: #### Verify based on presence of an element Define `DAST_AUTH_VERIFICATION_SELECTOR` as a [selector](#finding-an-elements-selector) that finds one or many elements on the page -displayed once the login form is successfully submitted. If no element is found, authentication is unsuccessful. +displayed after the login form is successfully submitted. If no element is found, authentication is unsuccessful. Searching for the selector on the page displayed when login fails should return no elements. For example: @@ -293,7 +334,7 @@ dast: #### Verify based on absence of a login form Define `DAST_AUTH_VERIFICATION_LOGIN_FORM` as `"true"` to indicate that DAST should search for the login form on the -page displayed once the login form is successfully submitted. If a login form is still present after logging in, authentication is unsuccessful. +page displayed after the login form is successfully submitted. If a login form is still present after logging in, authentication is unsuccessful. For example: diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index bdc08988cc0..7b263e5817d 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -42,10 +42,10 @@ To crawl a navigation path, DAST opens a browser window and instructs it to perf When the browser has finished loading the result of the final action, DAST inspects the page for actions a user might take, creates a new navigation for each found, and adds them to the navigation path to form new navigation paths. For example: -- DAST processes navigation path `LoadURL[https://example.com]`. -- DAST finds two user actions, `LeftClick[class=menu]` and `LeftClick[id=users]`. -- DAST creates two new navigation paths, `LoadURL[https://example.com] -> LeftClick[class=menu]` and `LoadURL[https://example.com] -> LeftClick[id=users]`. -- Crawling begins on the two new navigation paths. +1. DAST processes navigation path `LoadURL[https://example.com]`. +1. DAST finds two user actions, `LeftClick[class=menu]` and `LeftClick[id=users]`. +1. DAST creates two new navigation paths, `LoadURL[https://example.com] -> LeftClick[class=menu]` and `LoadURL[https://example.com] -> LeftClick[id=users]`. +1. Crawling begins on the two new navigation paths. It's common for an HTML element to exist in multiple places in an application, such as a menu visible on every page. Duplicate elements can cause crawlers to crawl the same pages again or become stuck in a loop. @@ -170,13 +170,15 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_ADVERTISE_SCAN` | boolean | `true` | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. | | `DAST_BROWSER_ACTION_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `800ms` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis after completing an action. | | `DAST_BROWSER_ACTION_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to complete an action. | -| `DAST_BROWSER_ALLOWED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. Headers set using `DAST_REQUEST_HEADERS` are added to every request made to these hostnames. | +| `DAST_BROWSER_ALLOWED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. Headers set using `DAST_REQUEST_HEADERS` are added to every request made to these hostnames. | | `DAST_BROWSER_COOKIES` | dictionary | `abtesting_group:3,region:locked` | A cookie name and value to be added to every request. | -| `DAST_BROWSER_CRAWL_GRAPH` | boolean | `true` | Set to `true` to generate an SVG graph of navigation paths visited during crawl phase of the scan. | -| `DAST_BROWSER_DEVTOOLS_LOG` | string | `Default:messageAndBody,truncate:2000` | Set to log protocol messages between DAST and the Chromium browser. | | +| `DAST_BROWSER_CRAWL_GRAPH` | boolean | `true` | Set to `true` to generate an SVG graph of navigation paths visited during crawl phase of the scan. You must also define `gl-dast-crawl-graph.svg` as a CI job artifact to be able to access the generated graph. | +| `DAST_BROWSER_CRAWL_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `5m` | The maximum amount of time to wait for the crawl phase of the scan to complete. Defaults to `24h`. | +| `DAST_BROWSER_DEVTOOLS_LOG` | string | `Default:messageAndBody,truncate:2000` | Set to log protocol messages between DAST and the Chromium browser. | +| `DAST_BROWSER_DOM_READY_AFTER_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `200ms` | Define how long to wait for updates to the DOM before checking a page is stable. Defaults to `500ms`. | | `DAST_BROWSER_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `600ms` | The maximum amount of time to wait for an element before determining it is ready for analysis. | | `DAST_BROWSER_EXCLUDED_ELEMENTS` | selector | `a[href='2.html'],css:.no-follow` | Comma-separated list of selectors that are ignored when scanning. | -| `DAST_BROWSER_EXCLUDED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered excluded and connections are forcibly dropped. | +| `DAST_BROWSER_EXCLUDED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered excluded and connections are forcibly dropped. | | `DAST_BROWSER_EXTRACT_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `5s` | The maximum amount of time to allow the browser to extract newly found elements or navigations. | | `DAST_BROWSER_FILE_LOG` | List of strings | `brows:debug,auth:debug` | A list of modules and their intended logging level for use in the file log. | | `DAST_BROWSER_FILE_LOG_PATH` | string | `/output/browserker.log` | Set to the path of the file log. | @@ -190,19 +192,21 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_BROWSER_NAVIGATION_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis after a navigation completes. | | `DAST_BROWSER_NAVIGATION_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `15s` | The maximum amount of time to wait for a browser to navigate from one page to another. | | `DAST_BROWSER_NUMBER_OF_BROWSERS` | number | `3` | The maximum number of concurrent browser instances to use. For shared runners on GitLab.com, we recommended a maximum of three. Private runners with more resources may benefit from a higher number, but are likely to produce little benefit after five to seven instances. | -| `DAST_BROWSER_PAGE_LOADING_SELECTOR` | selector | `css:#page-is-loading` | Selector that when is no longer visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_READY_SELECTOR` | -| `DAST_BROWSER_PAGE_READY_SELECTOR` | selector | `css:#page-is-ready` | Selector that when detected as visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_LOADING_SELECTOR` | +| `DAST_BROWSER_PAGE_LOADING_SELECTOR` | selector | `css:#page-is-loading` | Selector that when is no longer visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_READY_SELECTOR`. | +| `DAST_BROWSER_PAGE_READY_SELECTOR` | selector | `css:#page-is-ready` | Selector that when detected as visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_LOADING_SELECTOR`. | +| `DAST_BROWSER_PASSIVE_CHECK_WORKERS` | int | `5` | Number of workers that passive scan in parallel. Recommend setting to the number of available CPUs. | | `DAST_BROWSER_SCAN` | boolean | `true` | Required to be `true` to run a browser-based scan. | | `DAST_BROWSER_SEARCH_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `3s` | The maximum amount of time to allow the browser to search for new elements or user actions. | | `DAST_BROWSER_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis. | | `DAST_EXCLUDE_RULES` | string | `10020,10026` | Set to a comma-separated list of ZAP Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). | | `DAST_EXCLUDE_URLS` | URLs | `https://example.com/.*/sign-out` | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | +| `DAST_FF_ENABLE_BAS` | boolean | `true` | Set to `true` to [enable Breach and Attack Simulation](../breach_and_attack_simulation/index.md#extend-dynamic-application-security-testing-dast) during this DAST scan. | | `DAST_FULL_SCAN_ENABLED` | boolean | `true` | Set to `true` to run both passive and active checks. Default: `false` | | `DAST_PATHS` | string | `/page1.html,/category1/page3.html` | Set to a comma-separated list of URL paths relative to `DAST_WEBSITE` for DAST to scan. | | `DAST_PATHS_FILE` | string | `/builds/project/urls.txt` | Set to a file path containing a list of URL paths relative to `DAST_WEBSITE` for DAST to scan. The file must be plain text with one path per line. | | `DAST_PKCS12_CERTIFICATE_BASE64` | string | `ZGZkZ2p5NGd...` | The PKCS12 certificate used for sites that require Mutual TLS. Must be encoded as base64 text. | -| `DAST_PKCS12_PASSWORD` | string | `password` | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. Create sensitive [custom CI/CI variables](../../../ci/variables/index.md#define-a-cicd-variable-in-the-ui) using the GitLab UI. | -| `DAST_REQUEST_HEADERS` | string | `Cache-control:no-cache` | Set to a comma-separated list of request header names and values. | +| `DAST_PKCS12_PASSWORD` | string | `password` | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. Create sensitive [custom CI/CI variables](../../../ci/variables/index.md#define-a-cicd-variable-in-the-ui) using the GitLab UI. | +| `DAST_REQUEST_HEADERS` | string | `Cache-control:no-cache` | Set to a comma-separated list of request header names and values. | | `DAST_SKIP_TARGET_CHECK` | boolean | `true` | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. | | `DAST_TARGET_AVAILABILITY_TIMEOUT` | number | `60` | Time limit in seconds to wait for target availability. | | `DAST_WEBSITE` | URL | `https://example.com` | The URL of the website to scan. | @@ -232,9 +236,14 @@ This can come at a cost of increased scan time. You can manage the trade-off between coverage and scan time with the following measures: +- Vertically scale the runner and use a higher number of browsers with the [variable](#available-cicd-variables) `DAST_BROWSER_NUMBER_OF_BROWSERS`. The default is `3`. - Limit the number of actions executed by the browser with the [variable](#available-cicd-variables) `DAST_BROWSER_MAX_ACTIONS`. The default is `10,000`. - Limit the page depth that the browser-based crawler checks coverage on with the [variable](#available-cicd-variables) `DAST_BROWSER_MAX_DEPTH`. The crawler uses a breadth-first search strategy, so pages with smaller depth are crawled first. The default is `10`. -- Vertically scale the runner and use a higher number of browsers with [variable](#available-cicd-variables) `DAST_BROWSER_NUMBER_OF_BROWSERS`. The default is `3`. +- Limit the time taken to crawl the target application with the [variable](#available-cicd-variables) `DAST_BROWSER_CRAWL_TIMEOUT`. The default is `24h`. Scans continue with passive and active checks when the crawler times out. +- Build the crawl graph with the [variable](#available-cicd-variables) `DAST_BROWSER_CRAWL_GRAPH` to see what pages are being crawled. +- Prevent pages from being crawled using the [variable](#available-cicd-variables) `DAST_EXCLUDE_URLS`. +- Prevent elements being selected using the [variable](#available-cicd-variables) `DAST_BROWSER_EXCLUDED_ELEMENTS`. Use with caution, as defining this variable causes an extra lookup for each page crawled. +- If the target application has minimal or fast rendering, consider reducing the [variable](#available-cicd-variables) `DAST_BROWSER_DOM_READY_AFTER_TIMEOUT` to a smaller value. The default is `500ms`. ## Timeouts @@ -275,16 +284,6 @@ dast: NOTE: Adjusting these values may impact scan time because they adjust how long each browser waits for various activities to complete. -## Artifacts - -Using the latest version of the DAST [template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.latest.gitlab-ci.yml) these artifacts are exposed for download by default. - -The list of artifacts includes the following files: - -- `gl-dast-debug-auth-report.html` -- `gl-dast-debug-crawl-report.html` -- `gl-dast-crawl-graph.svg` - ## Troubleshooting See [troubleshooting](browser_based_troubleshooting.md) for more information. diff --git a/doc/user/application_security/dast/checks/16.7.md b/doc/user/application_security/dast/checks/16.7.md index d407234d2c2..edaace407ae 100644 --- a/doc/user/application_security/dast/checks/16.7.md +++ b/doc/user/application_security/dast/checks/16.7.md @@ -22,8 +22,8 @@ Only three directives are applicable for the `Strict-Transport-Security` header. 1. `includeSubDomains`: This optional, valueless directive signals that the policy applies to this host as well as any subdomains found under this host's domain. 1. `preload`: While not part of the specification, setting this optional value allows major browser organizations to add this site into the browser's preloaded set of HTTPS sites. This requires further action on behalf of the website operator to submit their domain to the browser's HSTS preload list. See [hstspreload.org](https://hstspreload.org/) for more information. -Note that invalid directives, or the `Strict-Transport-Security` header appearing more than once (if the values are -different) is considered invalid. +Invalid directives, or the `Strict-Transport-Security` header appearing more than once (if the +values are different) is considered invalid. Prior to adding to this security configuration to your website, it is recommended you review the hstspreload.org [Deployment Recommendations](https://hstspreload.org/#deployment-recommendations). diff --git a/doc/user/application_security/dast/dast_troubleshooting.md b/doc/user/application_security/dast/dast_troubleshooting.md index da382920604..c2e7f153e02 100644 --- a/doc/user/application_security/dast/dast_troubleshooting.md +++ b/doc/user/application_security/dast/dast_troubleshooting.md @@ -70,7 +70,7 @@ tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/20 ## Getting warning message `gl-dast-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ## Getting error `dast job: chosen stage does not exist` when including DAST CI template diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 3cb8967afd2..1f4506a22e5 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -15,6 +15,9 @@ visible from the source code. Dynamic Application Security Testing (DAST) examines applications for vulnerabilities like these in deployed environments. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Dynamic Application Security Testing (DAST)](https://www.youtube.com/watch?v=nbeDUoLZJTo). + NOTE: To learn how four of the top six attacks were application-based and how to protect your organization, download our diff --git a/doc/user/application_security/dast/proxy-based.md b/doc/user/application_security/dast/proxy-based.md index f70afac4c26..f65501712cc 100644 --- a/doc/user/application_security/dast/proxy-based.md +++ b/doc/user/application_security/dast/proxy-based.md @@ -80,10 +80,10 @@ To enable DAST to run automatically, either: - Enable [Auto DAST](../../../topics/autodevops/stages.md#auto-dast) (provided by [Auto DevOps](../../../topics/autodevops/index.md)). -- [Edit the `.gitlab.ci.yml` file manually](#edit-the-gitlabciyml-file-manually). -- [Use an automatically configured merge request](#configure-dast-using-the-ui). +- [Edit the `.gitlab-ci.yml` file manually](#edit-the-gitlab-ciyml-file-manually). +- [Configure DAST using the UI](#configure-dast-using-the-ui). -#### Edit the `.gitlab.ci.yml` file manually +#### Edit the `.gitlab-ci.yml` file manually In this method you manually edit the existing `.gitlab-ci.yml` file. Use this method if your GitLab CI/CD configuration file is complex. @@ -358,37 +358,36 @@ including a large number of false positives. | CI/CD variable | Type | Description | |:------------------------------------------------|:--------------|:------------------------------| -| `DAST_ADVERTISE_SCAN` | boolean | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. | -| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. | +| `DAST_ADVERTISE_SCAN` | boolean | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. | +| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. | | `DAST_ALLOWED_HOSTS` | Comma-separated list of strings | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. Headers set using `DAST_REQUEST_HEADERS` are added to every request made to these hostnames. Example, `site.com,another.com`. | -| `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 15.7. Replaced by [DAST API scan](../dast_api/index.md#available-cicd-variables). Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080`. | -| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 15.7. Replaced by [DAST API scan](../dast_api/index.md#available-cicd-variables). The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 16.0. Replaced by [DAST API scan](../dast_api/index.md#available-cicd-variables). | +| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 16.0. Replaced by [DAST API scan](../dast_api/index.md#available-cicd-variables). | | `DAST_AUTH_EXCLUDE_URLS` | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. | | `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false`. | -| `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. | +| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. | | `DAST_EXCLUDE_URLS` <sup>1</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Example, `http://example.com/sign-out`. | -| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293595)** in GitLab 14.0. Set to `true` to require domain validation when running DAST full scans. Default: `false` | | `DAST_FULL_SCAN_ENABLED` <sup>1</sup> | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | -| `DAST_HTML_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_MARKDOWN_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | -| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. [Introduced](https://gitlab.com/gitlab-org/security-products/dast/-/merge_requests/433) in GitLab 13.12. | -| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. | -| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. | -| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. | +| `DAST_HTML_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the HTML report written at the end of a scan. | +| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. | +| `DAST_MARKDOWN_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the Markdown report written at the end of a scan. | +| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked. Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | +| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. | +| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. | +| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. | +| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. | | `DAST_PKCS12_CERTIFICATE_BASE64` | string | The PKCS12 certificate used for sites that require Mutual TLS. Must be encoded as base64 text. | | `DAST_PKCS12_PASSWORD` | string | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. | | `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | -| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. | -| `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. | +| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. | +| `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. | +| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. | | `DAST_TARGET_AVAILABILITY_TIMEOUT` <sup>1</sup> | number | Time limit in seconds to wait for target availability. | -| `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_XML_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. | +| `DAST_XML_REPORT` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/384340)** in GitLab 15.7. The filename of the XML report written at the end of a scan. | | `DAST_WEBSITE` <sup>1</sup> | URL | The URL of the website to scan. | -| `DAST_ZAP_CLI_OPTIONS` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 15.7. ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_ZAP_CLI_OPTIONS` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 15.7. ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. | | `DAST_ZAP_LOG_CONFIGURATION` | string | **{warning}** **[Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383467)** in GitLab 15.7. Set to a semicolon-separated list of additional log4j properties for the ZAP Server. Example: `logger.httpsender.name=org.parosproxy.paros.network.HttpSender;logger.httpsender.level=debug;logger.sitemap.name=org.parosproxy.paros.model.SiteMap;logger.sitemap.level=debug;` | | `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. | diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index 0cdbb879cfc..d494259ecc4 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -48,6 +48,7 @@ The following projects demonstrate DAST API scanning: - [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/postman-example) - [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/graphql-example) - [Example SOAP project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/soap-example) +- [Authentication Token using Selenium](https://gitlab.com/gitlab-org/security-products/demos/api-dast/auth-token-selenium) ## Targeting API for DAST scanning @@ -97,8 +98,6 @@ The environment variables `DAST_API_OPENAPI_ALL_MEDIA_TYPES` and `DAST_API_OPENA #### Configure DAST API with an OpenAPI Specification -To configure DAST API scanning with an OpenAPI specification: - To configure DAST API scanning with an OpenAPI Specification: 1. [Include](../../../ci/yaml/index.md#includetemplate) @@ -264,6 +263,8 @@ This example is a minimal configuration for DAST API. From here you can: #### DAST API scanning with a GraphQL Schema file +DAST API can use a GraphQL schema file to understand and test a GraphQL endpoint that has introspection disabled. To use a GraphQL schema file, it must be in the introspection JSON format. A GraphQL schema can be converted to a the introspection JSON format using an online 3rd party tool: [https://transform.tools/graphql-to-introspection-json](https://transform.tools/graphql-to-introspection-json). + To configure DAST API to use a GraphQL schema file that provides information about the target API to test: 1. [Include](../../../ci/yaml/index.md#includetemplate) @@ -997,6 +998,14 @@ repository's root as `.gitlab/gitlab-dast-api-config.yml`. The following profiles are pre-defined in the default configuration file. Profiles can be added, removed, and modified by creating a custom configuration. +#### Passive + +- Application Information Check +- Cleartext Authentication Check +- JSON Hijacking Check +- Sensitive Information Check +- Session Cookie Check + #### Quick - Application Information Check @@ -1924,7 +1933,7 @@ variables: When configured correctly, a CI/CD pipeline contains a `dast` stage and an `dast_api` job. The job only fails when an invalid configuration is provided. During typical operation, the job always succeeds even if vulnerabilities are identified during testing. -Vulnerabilities are displayed on the **Security** pipeline tab with the suite name. When testing against the repositories default branch, the DAST API vulnerabilities are also shown on the Security & Compliance's Vulnerability Report page. +Vulnerabilities are displayed on the **Security** pipeline tab with the suite name. When testing against the repositories default branch, the DAST API vulnerabilities are also shown on the Security and Compliance's Vulnerability Report page. To prevent an excessive number of reported vulnerabilities, the DAST API scanner limits the number of vulnerabilities it reports per operation. @@ -1941,7 +1950,7 @@ Follow these steps to view details of a vulnerability: 1. You can view vulnerabilities in a project, or a merge request: - - In a project, go to the project's **Security & Compliance > Vulnerability Report** + - In a project, go to the project's **Security and Compliance > Vulnerability Report** page. This page shows all vulnerabilities from the default branch only. - In a merge request, go the merge request's **Security** section and select the **Expand** button. DAST API vulnerabilities are available in a section labeled @@ -2243,9 +2252,12 @@ dast_api_v1: variables: DAST_API_EXCLUDE_PATHS: /api/v1/** rules: - - if: $DAST_API_DISABLED + - if: $DAST_API_DISABLED == 'true' || $DAST_API_DISABLED == '1' when: never - - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH && + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == 'true' && + $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME + when: never + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == '1' && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - if: $CI_COMMIT_BRANCH && @@ -2258,9 +2270,12 @@ dast_api_v2: variables: DAST_API_EXCLUDE_PATHS: /api/v2/** rules: - - if: $DAST_API_DISABLED + - if: $DAST_API_DISABLED == 'true' || $DAST_API_DISABLED == '1' + when: never + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == 'true' && + $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH && + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == '1' && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - if: $CI_COMMIT_BRANCH && @@ -2292,9 +2307,12 @@ dast_api_branch: variables: DAST_API_EXCLUDE_PATHS: /api/large_response_json rules: - - if: $DAST_API_DISABLED + - if: $DAST_API_DISABLED == 'true' || $DAST_API_DISABLED == '1' when: never - - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH && + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == 'true' && + $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME + when: never + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == '1' && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - if: $CI_COMMIT_BRANCH && @@ -2310,9 +2328,12 @@ dast_api_branch: dast_api_main: extends: dast_api rules: - - if: $DAST_API_DISABLED + - if: $DAST_API_DISABLED == 'true' || $DAST_API_DISABLED == '1' + when: never + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == 'true' && + $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH && + - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH == '1' && $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME when: never - if: $CI_COMMIT_BRANCH && @@ -2553,14 +2574,75 @@ You can expedite the investigation with the [testssl.sh tool](https://testssl.sh 1. Run `./testssl.sh --log https://specs`. 1. Attach the log file to your support ticket. +### `ERROR: Job failed: failed to pull image` + +This error message occurs when pulling an image from a container registry that requires authentication to access (it is not public). + +In the job console output the error looks like: + +```log +Running with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a) + on blue-2.shared.runners-manager.gitlab.com/default XxUrkriX +Resolving secrets +00:00 +Preparing the "docker+machine" executor +00:06 +Using Docker executor with image registry.gitlab.com/security-products/api-security:2 ... +Starting service registry.example.com/my-target-app:latest ... +Pulling docker image registry.example.com/my-target-app:latest ... +WARNING: Failed to pull image with policy "always": Error response from daemon: Get https://registry.example.com/my-target-app/manifests/latest: unauthorized (manager.go:237:0s) +ERROR: Job failed: failed to pull image "registry.example.com/my-target-app:latest" with specified policies [always]: Error response from daemon: Get https://registry.example.com/my-target-app/manifests/latest: unauthorized (manager.go:237:0s) +``` + +**Error message** + +- In GitLab 15.9 and earlier, `ERROR: Job failed: failed to pull image` followed by `Error response from daemon: Get IMAGE: unauthorized`. + +**Solution** + +Authentication credentials are provided using the methods outlined in the [Access an image from a private Container Registry](../../../ci/docker/using_docker_images.md#access-an-image-from-a-private-container-registry) documentation section. The method used is dictated by your container registry provider and its configuration. If your using a container registry provided by a 3rd party, such as a cloud provider (Azure, Google Could (GCP), AWS and so on), check the providers documentation for information on how to authenticate to their container registries. + +The following example uses the [statically defined credentials](../../../ci/docker/using_docker_images.md#use-statically-defined-credentials) authentication method. In this example the container registry is `registry.example.com` and image is `my-target-app:latest`. + +1. Read how to [Determine your `DOCKER_AUTH_CONFIG` data](../../../ci/docker/using_docker_images.md#determine-your-docker_auth_config-data) to understand how to compute the variable value for `DOCKER_AUTH_CONFIG`. The configuration variable `DOCKER_AUTH_CONFIG` contains the Docker JSON configuration to provide the appropriate authentication information. For example, to access private container registry: `registry.example.com` with the credentials `aGVsbG8gd29ybGQK`, the Docker JSON looks like: + + ```json + { + "auths": { + "registry.example.com": { + "auth": "aGVsbG8gd29ybGQK" + } + } + } + ``` + +1. Add the `DOCKER_AUTH_CONFIG` as a CI/CD variable. Instead of adding the configuration variable directly in your `.gitlab-ci.yml`file you should create a project [CI/CD variable](../../../ci/variables/index.md#for-a-project). +1. Rerun your job, and the statically-defined credentials are now used to sign in to the private container registry `registry.example.com`, and let you pull the image `my-target-app:latest`. If succeeded the job console shows an output like: + + ```log + Running with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a) + on blue-4.shared.runners-manager.gitlab.com/default J2nyww-s + Resolving secrets + 00:00 + Preparing the "docker+machine" executor + 00:56 + Using Docker executor with image registry.gitlab.com/security-products/api-security:2 ... + Starting service registry.example.com/my-target-app:latest ... + Authenticating with credentials from $DOCKER_AUTH_CONFIG + Pulling docker image registry.example.com/my-target-app:latest ... + Using docker image sha256:139c39668e5e4417f7d0eb0eeb74145ba862f4f3c24f7c6594ecb2f82dc4ad06 for registry.example.com/my-target-app:latest with digest registry.example.com/my-target- + app@sha256:2b69fc7c3627dbd0ebaa17674c264fcd2f2ba21ed9552a472acf8b065d39039c ... + Waiting for services to be up and running (timeout 30 seconds)... + ``` + ## Get support or request an improvement To get support for your particular problem, use the [getting help channels](https://about.gitlab.com/get-help/). The [GitLab issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues) is the right place for bugs and feature proposals about API Security and DAST API. -Use `~"Category:API Security"` [label](../../../development/contributing/issue_workflow.md#labels) when opening a new issue regarding DAST API to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response. +Use `~"Category:API Security"` [label](../../../development/labels/index.md) when opening a new issue regarding DAST API to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response. -[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an award emoji and or join the discussion. +[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an emoji reaction or join the discussion. When experiencing a behavior not working as expected, consider providing contextual information: diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index b86d98471ac..afed5b3b0ca 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -13,11 +13,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use the dependency list to review your project's dependencies and key details about those dependencies, including their known vulnerabilities. It is a collection of dependencies in your project, including existing and new findings. -To see the dependency list, go to your project and select **Security & Compliance > Dependency list**. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Project Dependency](https://www.youtube.com/watch?v=ckqkn9Tnbw4). -This information is sometimes referred to as a Software Bill of Materials, SBOM, or BOM. +To see the dependency list, go to your project and select **Security and Compliance > Dependency list**. -The dependency list only shows the results of the last successful pipeline to run on the default branch. This is why we recommend not changing the default behavior of allowing the secure jobs to fail. +This information is sometimes referred to as a Software Bill of Materials, SBOM, or BOM. ## Prerequisites @@ -29,6 +30,9 @@ To view your project's dependencies, ensure you meet the following requirements: - Your project uses at least one of the [languages and package managers](../dependency_scanning/index.md#supported-languages-and-package-managers) supported by Gemnasium. +- A successful pipeline was run on the default branch. + You should not change the default behavior of allowing the + [application security jobs](../../application_security/index.md#application-coverage) to fail. ## View a project's dependencies diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 579dd4dfc4f..c510be55981 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -35,6 +35,9 @@ vulnerability. ![Dependency scanning Widget](img/dependency_scanning_v13_2.png) +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o). + ## Dependency Scanning compared to Container Scanning GitLab offers both Dependency Scanning and Container Scanning @@ -147,8 +150,8 @@ table.supported-languages ul { <thead> <tr> <th>Language</th> - <th>Language Versions</th> - <th>Package Manager</th> + <th>Language versions</th> + <th>Package manager</th> <th>Supported files</th> <th><a href="#how-multiple-files-are-processed">Processes multiple files?</a></th> </tr> @@ -187,14 +190,10 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td rowspan="2">Java</td> + <td rowspan="2">Java and Kotlin (not Android)<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup></td> <td rowspan="2"> 8 LTS, 11 LTS, - 13<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup>, - 14<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup>, - 15<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup>, - 16<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup>, or 17 LTS </td> <td><a href="https://gradle.org/">Gradle</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-2">2</a></b></sup></td> @@ -212,8 +211,8 @@ table.supported-languages ul { <td>N</td> </tr> <tr> - <td rowspan="2">JavaScript and TypeScript</td> - <td>All versions</td> + <td rowspan="3">JavaScript and TypeScript</td> + <td rowspan="3">All versions</td> <td><a href="https://www.npmjs.com/">npm</a></td> <td> <ul> @@ -224,12 +223,16 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td>All versions</td> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> <td><code>yarn.lock</code></td> <td>Y</td> </tr> <tr> + <td><a href="https://pnpm.io/">pnpm</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-3">3</a></b></sup></td> + <td><code>pnpm-lock.yaml</code></td> + <td>Y</td> + </tr> + <tr> <td>PHP</td> <td>All versions</td> <td><a href="https://getcomposer.org/">Composer</a></td> @@ -238,7 +241,7 @@ table.supported-languages ul { </tr> <tr> <td rowspan="4">Python</td> - <td rowspan="4">3.9</td> + <td rowspan="4">3.9, 3.10<sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></td> <td><a href="https://setuptools.readthedocs.io/en/latest/">setuptools</a></td> <td><code>setup.py</code></td> <td>N</td> @@ -258,8 +261,8 @@ table.supported-languages ul { <td><a href="https://pipenv.pypa.io/en/latest/">Pipenv</a></td> <td> <ul> - <li><a href="https://pipenv.pypa.io/en/latest/basics/#example-pipfile-pipfile-lock"><code>Pipfile</code></a></li> - <li><a href="https://pipenv.pypa.io/en/latest/basics/#example-pipfile-pipfile-lock"><code>Pipfile.lock</code></a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></li> + <li><a href="https://pipenv.pypa.io/en/latest/pipfile/#example-pipfile"><code>Pipfile</code></a></li> + <li><a href="https://pipenv.pypa.io/en/latest/pipfile/#example-pipfile-lock"><code>Pipfile.lock</code></a></li> </ul> </td> <td>N</td> @@ -295,28 +298,30 @@ table.supported-languages ul { <li> <a id="notes-regarding-supported-languages-and-package-managers-1"></a> <p> - Support for these versions of Java is deprecated and is planned to be removed in the GitLab 16.0 release. Additionally, these versions of Java are not supported by the FIPS-enabled image of <code>gemnasium-maven</code>. Official support is limited to LTS versions only. Although it may be possible to use Dependency Scanning with other versions by building a custom dependency scanning image, this approach is not officially supported by GitLab. + Support for Kotlin projects for Android is tracked in <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/336866">issue 336866</a>. </p> </li> <li> <a id="notes-regarding-supported-languages-and-package-managers-2"></a> <p> - Although Gradle with Java 8 is supported, there are other issues such that Android project builds are not supported at this time. - See the backlog issue <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/336866">Android support for Dependency - Scanning (gemnasium-maven)</a> for more details. Also, Gradle is not supported when <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. + Gradle is not supported when <a href="https://docs.gitlab.com/ee/development/fips_compliance.html#enable-fips-mode">FIPS mode</a> is enabled. </p> </li> <li> - <a id="notes-regarding-supported-languages-and-package-managers-4"></a> + <a id="notes-regarding-supported-languages-and-package-managers-3"></a> <p> - The presence of a <code>Pipfile.lock</code> file alone will <i>not</i> trigger the analyzer; the presence of a <code>Pipfile</code> is - still required in order for the analyzer to be executed. However, if a <code>Pipfile.lock</code> file is found, it is used by - <code>Gemnasium</code> to scan the exact package versions listed in this file. + Support for <code>pnpm</code> lockfiles was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/336809">introduced in GitLab 15.11</a>. <code>pnpm</code> lockfiles do not store bundled dependencies, so the reported dependencies may differ from <code>npm</code> or <code>yarn</code>. </p> + </li> + <li> + <a id="notes-regarding-supported-languages-and-package-managers-4"></a> <p> - Support for <code>Pipfile.lock</code> files without requiring the presence of a <code>Pipfile</code> is tracked in - issue: <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/299294">Dependency Scanning of Pipfile.lock without - installing project dependencies</a>. + For support of <code>Python 3.10</code>, add the following stanza to the GitLab CI/CD configuration file. This specifies that the <code>Python 3.10</code> image is to be used, instead of the default <code>Python 3.9</code>. + <div class="language-yaml highlighter-rouge"> + <div class="highlight"> +<pre class="highlight"><code><span class="na">gemnasium-dependency_scanning</span><span class="pi">:</span> + <span class="na">image</span><span class="pi">:</span> + <span class="na">name</span><span class="pi">:</span> <span class="s">$CI_TEMPLATE_REGISTRY_HOST/security-products/gemnasium-python:4-python-3.10</span></code></pre></div></div> </p> </li> <li> @@ -355,7 +360,8 @@ The following package managers use lockfiles that GitLab analyzers are capable o | Go | Not applicable | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/gosum/default/go.sum) <sup><strong><a href="#notes-regarding-parsing-lockfiles-1">1</a></strong></sup> | | NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | | npm | v1, v2, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-2">2</a></b></sup> | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/default/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4), [9.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/npm/fixtures/lockfile-v3/simple/package-lock.json#L4) | -| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/default/yarn.lock#L2) | +| pnpm | v5.3, v5.4, v6 | [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-pnpm/default/pnpm-lock.yaml#L1), [8.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/pnpm/fixtures/v6/simple/pnpm-lock.yaml#L1) | +| yarn | v1, v2<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup>, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup> | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/classic/default/yarn.lock#L2), [2.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v2/default/yarn.lock), [3.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v3/default/yarn.lock) | | Poetry | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/python-poetry/default/poetry.lock) | <!-- markdownlint-disable MD044 --> @@ -373,6 +379,26 @@ The following package managers use lockfiles that GitLab analyzers are capable o Support for <code>lockfileVersion = 3</code> was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/365176">introduced</a> in GitLab 15.7. </p> </li> + <li> + <a id="notes-regarding-parsing-lockfiles-3"></a> + <p> + Support for Yarn <code>v2</code> and <code>v3</code> was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/263358">introduced in GitLab 15.11</a>. However, this feature is also available to versions of GitLab 15.0 and later. + </p> + <p> + The following features are not supported for Yarn <code>v2</code> or <code>v3</code>: + </p> + <ul> + <li> + <a href="https://yarnpkg.com/features/workspaces">workspaces</a> + </li> + <li> + <a href="https://yarnpkg.com/cli/patch">yarn patch</a> + </li> + </ul> + <p> + Yarn files that contain a patch, a workspace, or both, are still processed, but these features are ignored. + </p> + </li> </ol> <!-- markdownlint-enable MD044 --> @@ -398,7 +424,7 @@ To support the following package managers, the GitLab analyzers proceed in two s <li> <a id="exported-dependency-information-notes-1"></a> <p> - This test uses the default version of <code>maven</code> specified by the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v3.10.6/build/gemnasium-maven/debian/config/.tool-versions#L3">.tool-versions</a> file. + This test uses the default version of <code>maven</code> specified by the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v3.10.6/build/gemnasium-maven/debian/config/.tool-versions#L3">`.tool-versions`</a> file. </p> </li> <li> @@ -417,7 +443,7 @@ To support the following package managers, the GitLab analyzers proceed in two s By default, the analyzer uses Java 17 and Gradle 7.3.3. </p> <p> - For Java versions <code>8</code> and <code>11</code>, Gradle <code>6.7.1</code> is automatically selected, and for Java versions <code>13</code> to <code>17</code>, Gradle <code>7.3.3</code> is automatically selected. + For Java versions <code>8</code> and <code>11</code>, Gradle <code>6.7.1</code> is automatically selected, and for Java version <code>17</code>, Gradle <code>7.3.3</code> is automatically selected. </p> </li> <li> @@ -553,7 +579,7 @@ always take the latest dependency scanning artifact available. To enable Dependency Scanning in a project, you can create a merge request: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Dependency Scanning** row, select **Configure with a merge request**. 1. Review and merge the merge request to enable Dependency Scanning. @@ -626,6 +652,7 @@ The following variables allow configuration of global dependency scanning settin | `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). | | `DS_EXCLUDED_PATHS` | Exclude files and directories from the scan based on the paths. A comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. | | `DS_IMAGE_SUFFIX` | Suffix added to the image name. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10.) Automatically set to `"-fips"` when FIPS mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) | +| `DS_MAX_DEPTH` | Defines how many directory levels deep that the analyzer should search for supported files to scan. A value of `-1` scans all directories regardless of depth. Default: `2`. | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info`. | @@ -642,16 +669,17 @@ The following variables are used for configuring specific analyzers (used for a | `DS_REMEDIATE` | `gemnasium` | `"true"`, `"false"` in FIPS mode | Enable automatic remediation of vulnerable dependencies. Not supported in FIPS mode. | | `DS_REMEDIATE_TIMEOUT` | `gemnasium` | `5m` | Timeout for auto-remediation. | | `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. | -| `DS_JAVA_VERSION` | `gemnasium-maven` | `17` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. Available versions in FIPS-enabled image: `8`, `11`, `17`. | +| `DS_JAVA_VERSION` | `gemnasium-maven` | `17` | Version of Java. Available versions: `8`, `11`, `17`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | | `PIP_INDEX_URL` | `gemnasium-python` | `https://pypi.org/simple` | Base URL of Python Package Index. | | `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma-separated. **Warning:** Read [the following security consideration](#python-projects) when using this environment variable. | | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | +| `PIPENV_PYPI_MIRROR` | `gemnasium-python` | | If set, overrides the PyPi index used by Pipenv with a [mirror](https://github.com/pypa/pipenv/blob/v2022.1.8/pipenv/environments.py#L263). | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | -| `DS_INCLUDE_DEV_DEPENDENCIES` | `gemnasium` | `"true"` | When set to `"false"`, development dependencies and their vulnerabilities are not reported. Only NPM and Poetry projects are supported. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227861) in GitLab 15.1. | +| `DS_INCLUDE_DEV_DEPENDENCIES` | `gemnasium` | `"true"` | When set to `"false"`, development dependencies and their vulnerabilities are not reported. Only Composer, NPM, and Poetry projects are supported. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227861) in GitLab 15.1. | | `GOOS` | `gemnasium` | `"linux"` | The operating system for which to compile Go code. | | `GOARCH` | `gemnasium` | `"amd64"` | The architecture of the processor for which to compile Go code. | | `GOFLAGS` | `gemnasium` | | The flags passed to the `go build` tool. | @@ -718,39 +746,8 @@ Gemnasium scanning jobs automatically use FIPS-enabled image when FIPS mode is e To manually switch to FIPS-enabled images, set the variable `DS_IMAGE_SUFFIX` to `"-fips"`. -To ensure compliance with FIPS, the FIPS-enabled image of `gemnasium-maven` uses the OpenJDK packages for RedHat UBI. -As a result, it only supports Java 8, 11, and 17. - Dependency scanning for Gradle projects and auto-remediation for Yarn projects are not supported in FIPS mode. -## Interacting with the vulnerabilities - -Once a vulnerability is found, you can interact with it. Read more on how to -[address the vulnerabilities](../vulnerabilities/index.md). - -## Solutions for vulnerabilities - -Some vulnerabilities can be fixed by applying the solution that GitLab -automatically generates. Read more about the -[solutions for vulnerabilities](../vulnerabilities/index.md#resolve-a-vulnerability). - -## Security Dashboard - -The Security Dashboard is a good place to get an overview of all the security -vulnerabilities in your groups, projects and pipelines. Read more about the -[Security Dashboard](../security_dashboard/index.md). - -## Vulnerabilities database update - -For more information about the vulnerabilities database update, see the -[maintenance table](../index.md#vulnerability-scanner-maintenance). - -## Dependency List - -An additional benefit of dependency scanning is the ability to view your -project's dependencies and their known vulnerabilities. Read more about -the [Dependency List](../dependency_list/index.md). - ## Reports JSON format The dependency scanning tool emits a JSON report file. For more information, see the @@ -867,7 +864,7 @@ Here's an example dependency scanning report: ### CycloneDX Software Bill of Materials -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta). > - Generally available in GitLab 15.7. In addition to the [JSON report file](#reports-json-format), the [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) @@ -908,7 +905,7 @@ Then the Gemnasium scanner generates the following CycloneDX SBOMs: └── gl-sbom-go-go.cdx.json ``` -The CycloneDX SBOMs can be downloaded [the same way as other job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +You can download CycloneDX SBOMs [the same way as other job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). ### Merging multiple CycloneDX SBOMs @@ -926,10 +923,30 @@ include: merge cyclonedx sboms: stage: merge-cyclonedx-sboms image: - name: cyclonedx/cyclonedx-cli:0.24.0 + name: cyclonedx/cyclonedx-cli:0.24.2 entrypoint: [""] script: - - find . -name "gl-sbom-*.cdx.json" -exec /cyclonedx merge --output-file gl-sbom-all.cdx.json --input-files "{}" + + - apt-get update && apt-get install -y jq + - find . -name "gl-sbom-*.cdx.json" -exec cyclonedx merge --output-file gl-sbom-all.cdx.json --input-files "{}" + + # remove duplicates from merged file. See https://github.com/CycloneDX/cyclonedx-cli/issues/188 for details. + - | + jq '. | + { + "bomFormat": .bomFormat, + "specVersion": .specVersion, + "serialNumber": .serialNumber, + "version": .version, + "metadata": { + "tools": [ + (.metadata.tools | unique[]) + ] + }, + "components": [ + (.components | unique[]) + ] + }' "gl-sbom-all.cdx.json" > gl-sbom-all.cdx.json.tmp && mv gl-sbom-all.cdx.json.tmp gl-sbom-all.cdx.json + # optional: validate the merged sbom + - cyclonedx validate --input-version v1_4 --input-file gl-sbom-all.cdx.json artifacts: paths: - gl-sbom-all.cdx.json @@ -980,9 +997,9 @@ import the following default dependency scanning analyzer images from `registry. your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/gemnasium:3 -registry.gitlab.com/security-products/gemnasium-maven:3 -registry.gitlab.com/security-products/gemnasium-python:3 +registry.gitlab.com/security-products/gemnasium:4 +registry.gitlab.com/security-products/gemnasium-maven:4 +registry.gitlab.com/security-products/gemnasium-python:4 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -1078,6 +1095,24 @@ ensure that it can reach your private repository. Here is an example configurati setuptools.ssl_support.cert_paths = ['internal.crt'] ``` +#### Python (Pipenv) + +If running in a limited network connectivity environment, you must configure the `PIPENV_PYPI_MIRROR` +variable to use a private PyPi mirror. This mirror must contain both default and development dependencies. + +```yaml +variables: + PIPENV_PYPI_MIRROR: https://pypi.example.com/simple +``` + +<!-- markdownlint-disable MD044 --> +Alternatively, if it's not possible to use a private registry, you can load the required packages +into the Pipenv virtual environment cache. For this option, the project must check in the +`Pipfile.lock` into the repository, and load both default and development packages into the cache. +See the example [python-pipenv](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/41cc017bd1ed302f6edebcfa3bc2922f428e07b6/.gitlab-ci.yml#L20-42) +project for an example of how this can be done. +<!-- markdownlint-enable MD044 --> + ## Hosting a copy of the `gemnasium_db` advisory database The [`gemnasium_db`](https://gitlab.com/gitlab-org/security-products/gemnasium-db) Git repository is @@ -1188,7 +1223,7 @@ affected. Read more in ### Getting warning message `gl-dependency-scanning-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Limitation when using rules:exists diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index c2f1257f989..48b2b8c1f1a 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -119,7 +119,7 @@ that you can download and analyze. To enable IaC Scanning in a project, you can create a merge request: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Infrastructure as Code (IaC) Scanning** row, select **Configure with a merge request**. 1. Review and merge the merge request to enable IaC Scanning. @@ -153,6 +153,8 @@ To disable analyzer rules: - Find it in the [JSON report artifact](#reports-json-format). - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) and copy the alphanumeric identifier that's shown. The rule name is shown on the [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected. +After you merge the `sast-ruleset.toml` file to the default branch, existing findings for disabled rules are [automatically resolved](#automatic-vulnerability-resolution). + In the following example `sast-ruleset.toml` file, the disabled rules are assigned to the `kics` analyzer by matching the `type` and `value` of identifiers: @@ -243,7 +245,8 @@ kics-iac-sast: ## Automatic vulnerability resolution -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. Enabled by default on GitLab.com; disabled by default in self-managed. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. +> - Enabled by default in 15.10. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. To help you focus on the vulnerabilities that are still relevant, GitLab IaC Scanning automatically [resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: @@ -260,7 +263,7 @@ The IaC tool emits a JSON report file in the existing SAST report format. For mo [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). The JSON report file can be downloaded from the CI pipelines page, or the -pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). +pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/jobs/job_artifacts.md). ## Troubleshooting diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 492b200d22b..a3c512a813c 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -101,7 +101,7 @@ The following vulnerability scanners and their databases are regularly updated: | Secure scanning tool | Vulnerabilities database updates | |:----------------------------------------------------------------|:---------------------------------| -| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | +| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. GitLab monitors this job through an internal alert that tells the engineering team when the database becomes more than 48 hours old. For more information, see the [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | | [Dependency Scanning](dependency_scanning/index.md) | Relies on the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). It is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | | [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | | [Static Application Security Testing (SAST)](sast/index.md) | The source of scan rules depends on which [analyzer](sast/analyzers.md) is used for each [supported programming language](sast/index.md#supported-languages-and-frameworks). GitLab maintains a ruleset for the Semgrep-based analyzer and updates it regularly based on internal research and user feedback. For other analyzers, the ruleset is sourced from the upstream open-source scanner. Each analyzer is updated at least once per month if a relevant update is available. | @@ -240,9 +240,9 @@ reports are available to download. To download a report, select ### Ultimate -A merge request contains a security widget which displays a summary of the new results. New results are determined by comparing the findings of the merge request against the findings of the most recent completed pipeline (`success`, `failed`, `canceled` or `skipped`) for the latest commit in the target branch. +A merge request contains a security widget which displays a summary of the _new_ results. New results are determined by comparing the findings of the merge request against the findings of the most recent completed pipeline (`success`, `failed`, `canceled` or `skipped`) for the commit when the feature branch was created from the target branch. -If security scans have not run for the most recent completed pipeline in the target branch there is no base for comparison. The vulnerabilities from the merge request findings are listed as new in the merge request security widget. We recommend you run a scan of the `default` (target) branch before enabling feature branch scans for your developers. +If security scans have not run for the completed pipeline in the target branch when the feature branch was created, there is no base for comparison. The vulnerabilities from the merge request findings are listed as new in the merge request security widget. We recommend you run a scan of the `default` (target) branch before enabling feature branch scans for your developers. The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both new and existing findings. @@ -274,13 +274,12 @@ By default, the vulnerability report does not show vulnerabilities of `dismissed > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9928) in GitLab 12.2. > - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357300) the Vulnerability-Check feature in GitLab 15.0. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/397067) the License-Check feature in GitLab 16.0. You can enforce an additional approval for merge requests that would introduce one of the following security issues: - A security vulnerability. For more details, read [Scan result policies](policies/scan-result-policies.md). -- A software license compliance violation. For more details, read - [Enabling license approvals within a project](../compliance/license_check_rules.md#enabling-license-approvals-within-a-project). ## Using private Maven repositories @@ -577,7 +576,7 @@ Debug logging can be a serious security risk. The output may contain the content environment variables and other secrets available to the job. The output is uploaded to the GitLab server and visible in job logs. -This message is often followed by the [error `No files to upload`](../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload), +This message is often followed by the [error `No files to upload`](../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload), and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Check the entire job log for such messages. If you don't find these messages, retry the failed job after setting `SECURE_LOG_LEVEL: "debug"` as a [custom CI/CD variable](../../ci/variables/index.md#for-a-project). diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 5a075bf731b..1ee4d9b2a19 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -121,7 +121,7 @@ include: ``` The pipeline downloads the Docker images needed for the Security Scanners and saves them as -[job artifacts](../../../ci/pipelines/job_artifacts.md) or pushes them to the [Container Registry](../../packages/container_registry/index.md) +[job artifacts](../../../ci/jobs/job_artifacts.md) or pushes them to the [Container Registry](../../packages/container_registry/index.md) of the project where the pipeline is executed. These archives can be transferred to another location and [loaded](https://docs.docker.com/engine/reference/commandline/load/) in a Docker daemon. This method requires a runner with access to both `gitlab.com` (including diff --git a/doc/user/application_security/policies/img/association_diagram.png b/doc/user/application_security/policies/img/association_diagram.png Binary files differindex d082e297c68..3a56aeba91b 100644 --- a/doc/user/application_security/policies/img/association_diagram.png +++ b/doc/user/application_security/policies/img/association_diagram.png diff --git a/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png b/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png Binary files differdeleted file mode 100644 index 8ca7547a33c..00000000000 --- a/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/policy_rule_mode_v15_9.png b/doc/user/application_security/policies/img/policy_rule_mode_v15_9.png Binary files differnew file mode 100644 index 00000000000..8cb2e82ac05 --- /dev/null +++ b/doc/user/application_security/policies/img/policy_rule_mode_v15_9.png diff --git a/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png b/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png Binary files differdeleted file mode 100644 index 1d71e8684e9..00000000000 --- a/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/policy_yaml_mode_v15_9.png b/doc/user/application_security/policies/img/policy_yaml_mode_v15_9.png Binary files differnew file mode 100644 index 00000000000..95b637efef3 --- /dev/null +++ b/doc/user/application_security/policies/img/policy_yaml_mode_v15_9.png diff --git a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png Binary files differnew file mode 100644 index 00000000000..76b9a971172 --- /dev/null +++ b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_11.png diff --git a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_5.png b/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_5.png Binary files differdeleted file mode 100644 index 5ae7c2e065a..00000000000 --- a/doc/user/application_security/policies/img/scan_execution_policy_rule_mode_v15_5.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/scheduled_scan_execution_policies_diagram.png b/doc/user/application_security/policies/img/scheduled_scan_execution_policies_diagram.png Binary files differnew file mode 100644 index 00000000000..fcf7a8352fd --- /dev/null +++ b/doc/user/application_security/policies/img/scheduled_scan_execution_policies_diagram.png diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index a214d0d2cec..0d821f8e47c 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Policies in GitLab provide security teams a way to require scans of their choice to be run whenever a project pipeline runs according to the configuration specified. Security teams can therefore be confident that the scans they set up have not been changed, altered, or disabled. You -can access these by navigating to your project's **Security & Compliance > Policies** page. +can access these by navigating to your project's **Security and Compliance > Policies** page. GitLab supports the following security policies: @@ -23,9 +23,13 @@ GitLab supports the following security policies: ## Security policy project All security policies are stored as YAML in a separate security policy project that gets linked to -the development project. This association can be a one-to-many relationship, allowing one security -policy project to apply to multiple development projects. Linked projects are not required to be in -the same group as the development projects to which they are linked. +the development project, group, or sub-group. This association can be a one-to-many relationship, allowing one security +policy project to apply to multiple development projects, groups, or sub-groups: + +- For self-managed GitLab instances, linked projects are not required to be in the same group + or the same subgroup as the development projects to which they are linked. +- For GitLab SaaS, the security policy project is required to be in the same top-level group + as the development project, although, it is not necessary for the project to be in the same subgroup. ![Security Policy Project Linking Diagram](img/association_diagram.png) @@ -34,10 +38,6 @@ project and the security policy project, this is not recommended. Keeping the se project separate from the development project allows for complete separation of duties between security/compliance teams and development teams. -You should not link a security policy project to a development project and to the group -or sub-group the development project belongs to at the same time. Linking this way will result in -approval rules from the Scan Result Policy not being applied to merge requests in the development project. - All security policies are stored in the `.gitlab/security-policies/policy.yml` YAML file inside the linked security policy project. The format for this YAML is specific to the type of policy that is stored there. Examples and schema information are available for the following policy types: @@ -59,7 +59,7 @@ As a project owner, take the following steps to create or edit an association be project and a project that you would like to designate as the security policy project: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Policies**. +1. On the left sidebar, select **Security and Compliance > Policies**. 1. Select **Edit Policy Project**, and search for and select the project you would like to link from the dropdown list. 1. Select **Save**. @@ -83,7 +83,7 @@ policy's information (for example, description or enforcement status), and create and edit deployed policies: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Policies**. +1. On the left sidebar, select **Security and Compliance > Policies**. ![Policies List Page](img/policies_list_v15_1.png) @@ -94,7 +94,7 @@ status), and create and edit deployed policies: You can use the policy editor to create, edit, and delete policies: 1. On the top bar, select **Main menu > Projects** and find your group. -1. On the left sidebar, select **Security & Compliance > Policies**. +1. On the left sidebar, select **Security and Compliance > Policies**. - To create a new policy, select **New policy** which is located in the **Policies** page's header. You can then select which type of policy to create. - To edit an existing policy, select **Edit policy** in the selected policy drawer. @@ -104,13 +104,13 @@ The policy editor has two modes: - The visual _Rule_ mode allows you to construct and preview policy rules using rule blocks and related controls. - ![Policy Editor Rule Mode](img/policy_rule_mode_v14_9.png) + ![Policy Editor Rule Mode](img/policy_rule_mode_v15_9.png) - YAML mode allows you to enter a policy definition in `.yaml` format and is aimed at expert users and cases that the Rule mode doesn't support. - ![Policy Editor YAML Mode](img/policy_yaml_mode_v14_9.png) + ![Policy Editor YAML Mode](img/policy_yaml_mode_v15_9.png) You can use both modes interchangeably and switch between them at any time. If a YAML resource is incorrect or contains data not supported @@ -129,19 +129,6 @@ time that the first policy merge request is created. You can use the [Vulnerability-Check Migration](https://gitlab.com/gitlab-org/gitlab/-/snippets/2328089) script to bulk create policies or associate security policy projects with development projects. For instructions and a demonstration of how to use the Vulnerability-Check Migration script, see [this video](https://youtu.be/biU1N26DfBc). -## Scan execution policies - -See [Scan execution policies](scan-execution-policies.md). - -## Scan result policy editor - -See [Scan result policies](scan-result-policies.md). - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/govern/security_policies/security_policy_management/) -for more information on the product direction of security policies within GitLab. - ## Troubleshooting ### `Branch name 'update-policy-<timestamp>' does not follow the pattern '<branch_name_regex>'` @@ -151,3 +138,17 @@ When you create a new security policy or change an existing policy, a new branch If you have group or instance [push rules that do not allow branch name patterns](../../project/repository/push_rules.md#validate-branch-names) that contain the text `update-policy-<timestamp>`, you will get an error that states `Branch name 'update-policy-<timestamp>' does not follow the pattern '<branch_name_regex>'`. The workaround is to amend your group or instance push rules to allow branches following the pattern `update-policy-` followed by an integer timestamp. + +### Troubleshooting common issues configuring security policies + +- Confirm that projects contain a `.gitlab-ci.yml` file. This file is required for scan execution policies. +- Confirm that scanners are properly configured and producing results for the latest branch. Security Policies are designed to require approval when there are no results (no security report), as this ensures that no vulnerabilities are introduced. We cannot know if there are any vulnerabilities unless the scans enforced by the policy complete successfully and are evaluated. +- When running scan execution policies based on a SAST action, ensure target repositories contain proper code files. SAST runs different analyzers [based on the types of files in the repo](../sast/index.md#supported-languages-and-frameworks), and if no supported files are found it will not run any jobs. See the [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) for more details. +- Check for any branch configuration conflicts. If your policy is configured to enforce rules on `main` but some projects within the scope are using `master` as their default branch, the policy is not applied for the latter. Support for specifying the `default` branch in your policies is proposed in [epic 9468](https://gitlab.com/groups/gitlab-org/-/epics/9468). +- Scan result policies created at the group or sub-group level can take some time to apply to all the merge requests in the group. +- Scheduled scan execution policies run with a minimum 15 minute cadence. Learn more [about the schedule rule type](../policies/scan-execution-policies.md#schedule-rule-type). +- When scheduling pipelines, keep in mind that CRON scheduling is based on UTC on GitLab SaaS and is based on your server time for self managed instances. When testing new policies, it may appear pipelines are not running properly when in fact they are scheduled in your server's timezone. +- When enforcing scan execution policies, the target project's pipeline is triggered by the user who last updated the security policy project's `policy.yml` file. The user must have permission to trigger the pipeline in the project for the policy to be enforced, and the pipeline to run. Work to address this is being tracked in [issue 394958](https://gitlab.com/gitlab-org/gitlab/-/issues/394958). +- You should not link a security policy project to a development project and to the group or sub-group the development project belongs to at the same time. Linking this way will result in approval rules from the Scan Result Policy not being applied to merge requests in the development project. + +If you are still experiencing issues, you can [view recent reported bugs](https://gitlab.com/gitlab-org/gitlab/-/issues/?sort=popularity&state=opened&label_name%5B%5D=group%3A%3Asecurity%20policies&label_name%5B%5D=type%3A%3Abug&first_page_size=20) and raise new unreported issues. diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 26c7e1d9c77..61671efab63 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -33,7 +33,7 @@ NOTE: Only group, subgroup, or project Owners have the [permissions](../../permissions.md#project-members-permissions) to select Security Policy Project. -Once your policy is complete, save it by selecting **Create via merge request** +Once your policy is complete, save it by selecting **Configure with a merge request** at the bottom of the editor. You are redirected to the merge request on the project's configured security policy project. If one does not link to your project, a security policy project is automatically created. Existing policies can also be @@ -44,7 +44,7 @@ Most policy changes take effect as soon as the merge request is merged. Any chan do not go through a merge request and are committed directly to the default branch may require up to 10 minutes before the policy changes take effect. -![Scan Execution Policy Editor Rule Mode](img/scan_execution_policy_rule_mode_v15_5.png) +![Scan Execution Policy Editor Rule Mode](img/scan_execution_policy_rule_mode_v15_11.png) ## Scan execution policies schema @@ -88,7 +88,7 @@ This rule enforces the defined actions and schedules a scan on the provided date |------------|------|-----------------|-------------| | `type` | `string` | `schedule` | The rule's type. | | `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). This field is required if the `agents` field is not set. | -| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | +| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. Minimum of 15 minute intervals when used together with the `branches` field. | | `agents` | `object` | | The name of the [GitLab agents](../../clusters/agent/index.md) where [Operational Container Scanning](../../clusters/agent/vulnerabilities.md) runs. The object key is the name of the Kubernetes agent configured for your project in GitLab. This field is required if the `branches` field is not set. | GitLab supports the following types of CRON syntax for the `cadence` field: @@ -99,8 +99,18 @@ GitLab supports the following types of CRON syntax for the `cadence` field: NOTE: Other elements of the [CRON syntax](https://docs.oracle.com/cd/E12058_01/doc/doc.1014/e12030/cron_expressions.htm) may work in the cadence field if supported by the [cron](https://github.com/robfig/cron) we are using in our implementation, however, GitLab does not officially test or support them. -NOTE: -If using the `agents` field, required for `Operational Container Scanning`, the CRON expression is evaluated in [UTC](https://www.timeanddate.com/worldclock/timezone/utc) using the system-time of the Kubernetes-agent pod. If not using the `agents` field, the CRON expression is evaluated in standard [UTC](https://www.timeanddate.com/worldclock/timezone/utc) time from GitLab.com. If you have a self-managed GitLab instance and have [changed the server time zone](../../../administration/timezone.md), the CRON expression is evaluated with the new time zone. +When using the `schedule` rule type in conjunction with the `agents` field, note the following: + +- The GitLab Agent for Kubernetes checks every 30 seconds to see if there is an applicable policy. When a policy is found, the scans are executed according to the `cadence` defined. +- The CRON expression is evaluated using the system-time of the Kubernetes-agent pod. + +When using the `schedule` rule type in conjunction with the `branches` field, note the following: + +- The cron worker runs on 15 minute intervals and starts any pipelines that were scheduled to run during the previous 15 minutes. +- Based on your rule, you might expect scheduled pipelines to run with an offset of up to 15 minutes. +- The CRON expression is evaluated in standard [UTC](https://www.timeanddate.com/worldclock/timezone/utc) time from GitLab.com. If you have a self-managed GitLab instance and have [changed the server time zone](../../../administration/timezone.md), the CRON expression is evaluated with the new time zone. + +![CRON worker diagram](img/scheduled_scan_execution_policies_diagram.png) ### `agent` schema @@ -140,7 +150,7 @@ rule in the defined policy are met. | Field | Type | Possible values | Description | |-------|------|-----------------|-------------| -| `scan` | `string` | `dast`, `secret_detection`, `sast`, `container_scanning`, `dependency_scanning` | The action's type. | +| `scan` | `string` | `sast`, `sast_iac`, `dast`, `secret_detection`, `container_scanning`, `dependency_scanning` | The action's type. | | `site_profile` | `string` | Name of the selected [DAST site profile](../dast/proxy-based.md#site-profile). | The DAST site profile to execute the DAST scan. This field should only be set if `scan` type is `dast`. | | `scanner_profile` | `string` or `null` | Name of the selected [DAST scanner profile](../dast/proxy-based.md#scanner-profile). | The DAST scanner profile to execute the DAST scan. This field should only be set if `scan` type is `dast`.| | `variables` | `object` | | A set of CI variables, supplied as an array of `key: value` pairs, to apply and enforce for the selected scan. The `key` is the variable name, with its `value` provided as a string. This parameter supports any variable that the GitLab CI job supports for the specified scan. | @@ -247,5 +257,5 @@ developer may want to try running a SAST scan with different variables than the this case, two SAST jobs run in the pipeline, one with the developer's variables and one with the security and compliance team's variables. If you want to avoid running duplicate scans, you can either remove the scans from the project's `.gitlab-ci.yml` file or disable your -local jobs by setting `SAST_DISABLED: true`. Disabling jobs this way does not prevent the security jobs defined by scan execution +local jobs by setting `SAST_DISABLED: "true"`. Disabling jobs this way does not prevent the security jobs defined by scan execution policies from running. diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index bc74b8bdfb1..ecbbf4703b0 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -10,8 +10,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can use scan result policies to take action based on scan results. For example, one type of scan result policy is a security approval policy that allows approval to be required based on the -findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning -job is fully executed. The following video gives you an overview of GitLab scan result policies: +findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning job is fully executed. + +NOTE: +Scan result policies are applicable only to [protected](../../project/protected_branches.md) target branches. + +The following video gives you an overview of GitLab scan result policies: <div class="video-fallback"> See the video: <a href="https://youtu.be/w5I9gcUgr9U">Overview of GitLab Scan Result Policies</a>. @@ -29,7 +33,7 @@ NOTE: Only project Owners have the [permissions](../../permissions.md#project-members-permissions) to select Security Policy Project. -Once your policy is complete, save it by selecting **Create merge request** at the bottom of the +Once your policy is complete, save it by selecting **Configure with a merge request** at the bottom of the editor. This redirects you to the merge request on the project's configured security policy project. If a security policy project doesn't link to your project, GitLab creates such a project for you. Existing policies can also be removed from the editor interface by selecting **Delete policy** at @@ -72,22 +76,19 @@ the following sections and tables provide an alternative. This rule enforces the defined actions based on security scan findings. -| Field | Type | Possible values | Description | -|------------|------|-----------------|-------------| -| `type` | `string` | `scan_finding` | The rule's type. | -| `branches` | `array` of `string` | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. | -| `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. | -| `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | -| `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical`| The severity levels for this rule to consider. | -| `vulnerability_states` | `array` of `string` | `newly_detected`, `detected`, `confirmed`, `resolved`, `dismissed` | All vulnerabilities fall into two categories:<br><br>**Newly Detected Vulnerabilities** - the `newly_detected` policy option covers vulnerabilities identified in the merge request branch itself but that do not currently exist on the default branch. This policy option requires a pipeline to complete before the rule is evaluated so that it knows whether vulnerabilities are newly detected or not. Merge requests are blocked until the pipeline and necessary security scans are complete. The `newly_detected` option considers both of the following statuses:<br><br> • Detected<br> • Dismissed<br><br>**Pre-Existing Vulnerabilities** - these policy options are evaluated immediately and do not require a pipeline complete as they consider only vulnerabilities previously detected in the default branch.<br><br> • `Detected` - the policy looks for vulnerabilities in the detected state.<br> • `Confirmed` - the policy looks for vulnerabilities in the confirmed state.<br> • `Dismissed` - the policy looks for vulnerabilities in the dismissed state.<br> • `Resolved` - the policy looks for vulnerabilities in the resolved state. | +| Field | Type | Possible values | Description | +|------------|------|--------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `type` | `string` | `scan_finding` | The rule's type. | +| `branches` | `array` of `string` | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. | +| `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. `sast` includes results from both SAST and SAST IaC scanners. | +| `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | +| `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical` | The severity levels for this rule to consider. | +| `vulnerability_states` | `array` of `string` | `newly_detected`, `detected`, `confirmed`, `resolved`, `dismissed`, `new_needs_triage`, `new_dismissed` | All vulnerabilities fall into two categories:<br><br>**Newly Detected Vulnerabilities** - the `newly_detected` policy option covers vulnerabilities identified in the merge request branch itself but that do not currently exist on the default branch. This policy option requires a pipeline to complete before the rule is evaluated so that it knows whether vulnerabilities are newly detected or not. Merge requests are blocked until the pipeline and necessary security scans are complete. The `newly_detected` option considers both of the following statuses:<br><br> • Detected<br> • Dismissed<br><br> The `new_needs_triage` option considers the status<br><br> • Detected<br><br> The `new_dismissed` option considers the status<br><br> • Dismissed<br><br>**Pre-Existing Vulnerabilities** - these policy options are evaluated immediately and do not require a pipeline complete as they consider only vulnerabilities previously detected in the default branch.<br><br> • `Detected` - the policy looks for vulnerabilities in the detected state.<br> • `Confirmed` - the policy looks for vulnerabilities in the confirmed state.<br> • `Dismissed` - the policy looks for vulnerabilities in the dismissed state.<br> • `Resolved` - the policy looks for vulnerabilities in the resolved state. | ## `license_finding` rule type -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `license_scanning_policies`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `license_scanning_policies`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `license_scanning_policies`. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/397644) in GitLab 15.11. Feature flag `license_scanning_policies` removed. This rule enforces the defined actions based on license findings. @@ -112,6 +113,7 @@ the defined policy. | `user_approvers_ids` | `array` of `integer` | ID of one of more users | The IDs of users to consider as approvers. Users must have access to the project to be eligible to approve. | | `group_approvers` | `array` of `string` | Path of one of more groups | The groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | | `group_approvers_ids` | `array` of `integer` | ID of one of more groups | The IDs of groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | +| `role_approvers` | `array` of `string` | One or more [roles](../../../user/permissions.md#roles) (for example: `owner`, `maintainer`) | The roles to consider as approvers that are eligible to approve. | Requirements and limitations: @@ -119,6 +121,7 @@ Requirements and limitations: Otherwise, scan result policies do not have any effect. - The maximum number of policies is five. - Each policy can have a maximum of five rules. +- All configured scanners must be present in the merge request's latest pipeline. If not, approvals are required even if some vulnerability criteria have not been met. ## Example security scan result policies project @@ -167,6 +170,8 @@ scan_result_policy: approvals_required: 1 user_approvers: - sam.white + role_approvers: + - owner ``` In this example: diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index c8542142830..f52ff7f9a55 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -33,7 +33,6 @@ SAST supports the following official analyzers: - [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) (NodeJsScan) - [`phpcs-security-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP CS security-audit) - [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) (PMD (Apex only)) -- [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)) - [`semgrep`](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) (Semgrep) - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) @@ -43,6 +42,7 @@ SAST has used other analyzers in previous versions. These analyzers reached End - [`bandit`](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Bandit); [End of Support](https://gitlab.com/gitlab-org/gitlab/-/issues/352554) in GitLab 15.4. Replaced by the `semgrep` analyzer with GitLab-managed rules. - [`eslint`](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (ESLint (JavaScript and React)); [End of Support](https://gitlab.com/gitlab-org/gitlab/-/issues/352554) in GitLab 15.4. Replaced by the `semgrep` analyzer with GitLab-managed rules. - [`gosec`](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Gosec); [End of Support](https://gitlab.com/gitlab-org/gitlab/-/issues/352554) in GitLab 15.4. Replaced by the `semgrep` analyzer with GitLab-managed rules. +- [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)); [End of Support](https://gitlab.com/gitlab-org/gitlab/-/issues/390416) in GitLab 16.0. Replaced by the `semgrep` analyzer with GitLab-managed rules. ## SAST analyzer features @@ -252,7 +252,7 @@ Each analyzer provides data about the vulnerabilities it detects. The following data available from each analyzer. The values provided by these tools are heterogeneous so they are sometimes normalized into common values, for example, `severity` and `confidence`. -| Property / tool | Apex | Bandit<sup>1</sup> | Brakeman | ESLint security<sup>1</sup> | SpotBugs | Flawfinder | Gosec<sup>1</sup> | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Semgrep | Sobelow | +| Property / tool | Apex | Bandit<sup>1</sup> | Brakeman | ESLint security<sup>1</sup> | SpotBugs | Flawfinder | Gosec<sup>1</sup> | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET)<sup>1</sup> | Semgrep | Sobelow | |--------------------------------|------|--------|----------|-----------------|----------|------------|-------|-----------------|-------|------------|-----------------------|---------------------------|---------|---------| | Affected item (for example, class or package) | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | Confidence | ✗ | ✓ | ✓ | ✗ | ✓ | x | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✓ | diff --git a/doc/user/application_security/sast/customize_rulesets.md b/doc/user/application_security/sast/customize_rulesets.md index d070883df9a..ddf8db4e489 100644 --- a/doc/user/application_security/sast/customize_rulesets.md +++ b/doc/user/application_security/sast/customize_rulesets.md @@ -23,12 +23,18 @@ repository being scanned. There are two kinds of customization: ## Disable predefined rules -You can disable predefined rules for any SAST analyzer. Disabled rules don't appear -on the [Pipeline Security](../index.md#view-security-scan-information-in-the-pipeline-security-tab) -tab or the [Vulnerability Report](../index.md#view-security-scan-information-in-the-vulnerability-report). +You can disable predefined rules for any SAST analyzer. -Disabling rules has a retroactive effect. The analyzer continues to scan for the -vulnerability, but findings are omitted from the [`gl-sast-report.json` artifact](index.md#reports-json-format). +When you disable a rule: + +- Most analyzers still scan for the vulnerability. The results are removed as a processing step after the scan completes, and they don't appear in the [`gl-sast-report.json` artifact](index.md#reports-json-format). +- Findings for the disabled rule no longer appear in the [Pipeline Security tab](../index.md#view-security-scan-information-in-the-pipeline-security-tab). +- Existing findings for the disabled rule on the default branch are marked ["No longer detected"](../vulnerability_report/index.md#activity-filter) in the [Vulnerability Report](../index.md#view-security-scan-information-in-the-vulnerability-report). + +The Semgrep-based analyzer handles disabled rules differently: + +- To improve performance, the Semgrep-based analyzer doesn't scan for disabled rules at all. +- If you disable a rule in the Semgrep-based analyzer, existing vulnerability findings for that rule are [automatically resolved](index.md#automatic-vulnerability-resolution) after you merge the `sast-ruleset.toml` file to the default branch. See the [Schema](#schema) and [Examples](#examples) sections for information on how to configure this behavior. @@ -323,7 +329,7 @@ With the following custom ruleset configuration, vulnerabilities found with [semgrep] [[semgrep.ruleset]] [semgrep.ruleset.identifier] - type = "CWE" + type = "cwe" value = "322" [semgrep.ruleset.override] severity = "Critical" diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index b0d84e4cff9..64c0f3440c5 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -77,45 +77,46 @@ For more information about our plans for language support in SAST, see the [cate | Language / framework | [Analyzer](analyzers.md) used for scanning | Minimum supported GitLab version | |------------------------------|--------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| -| .NET Core | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 11.0 | -| .NET Framework<sup>1</sup> | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 13.0 | -| .NET (all versions, C# only) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 15.4 | +| .NET Core<sup>3</sup> | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 11.0 | +| .NET Framework<sup>3</sup> | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 13.0 | +| .NET (all versions, C# only) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 15.4 | | Apex (Salesforce) | [PMD](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) | 12.1 | -| C | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.2 | +| C | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.2 | | C/C++ | [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) | 11.1 | -| Go<sup>3</sup> | [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) | 10.7 | -| Go | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.4 | -| Groovy<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) | +| Go<sup>2</sup> | [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) | 10.7 | +| Go | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.4 | +| Groovy<sup>1</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) | | Helm Charts | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 13.1 | -| Java (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.10 | -| Java<sup>2, 3</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) | +| Java (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 14.10 | +| Java<sup>1, 2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) | | Java (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | -| JavaScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.8 | -| JavaScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| JavaScript<sup>2</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.8 | +| JavaScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | | Kotlin (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | -| Kotlin (General)<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 13.11 | +| Kotlin (General)<sup>1</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 13.11 | | Kubernetes manifests | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 12.6 | | Node.js | [NodeJsScan](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) | 11.1 | | Objective-C (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | | PHP | [phpcs-security-audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) | 10.8 | -| Python<sup>3</sup> | [bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) | 10.3 | -| Python | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.9 | -| React<sup>3</sup> | [ESLint react plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 12.5 | -| React | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| Python<sup>2</sup> | [bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) | 10.3 | +| Python | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.9 | +| React<sup>2</sup> | [ESLint react plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 12.5 | +| React | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | | Ruby | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 13.9 | | Ruby on Rails | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 10.3 | -| Scala<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) | +| Scala (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 16.0 | +| Scala<sup>1</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) | | Swift (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 | -| TypeScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 | -| TypeScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 | +| TypeScript<sup>2</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 | +| TypeScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with [GitLab-managed rules](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/blob/main/RULES.md) | 13.10 | -1. .NET 4 support is limited. The analyzer runs in a Linux container and does not have access to Windows-specific libraries or features. Use the Semgrep-based scanner if you need .NET 4 support. 1. The SpotBugs-based analyzer supports [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/), and [SBT](https://www.scala-sbt.org/). It can also be used with variants like the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html), [Grails](https://grails.org/), -and the [Maven wrapper](https://github.com/takari/maven-wrapper). However, SpotBugs has [limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/350801) when used against [Ant](https://ant.apache.org/)-based projects. We recommend using the Semgrep-based analyzer for Ant-based Java projects. +and the [Maven wrapper](https://github.com/takari/maven-wrapper). However, SpotBugs has [limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/350801) when used against [Ant](https://ant.apache.org/)-based projects. We recommend using the Semgrep-based analyzer for Ant-based Java or Scala projects. 1. These analyzers reached [End of Support](https://about.gitlab.com/handbook/product/gitlab-the-product/#end-of-support) status [in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/352554). +1. Security Code Scan reached [End of Support](https://about.gitlab.com/handbook/product/gitlab-the-product/#end-of-support) status [in GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/390416). ## Multi-project support @@ -183,7 +184,8 @@ For more information, see the confidential project `https://gitlab.com/gitlab-or ## Automatic vulnerability resolution -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. Enabled by default on GitLab.com; disabled by default in self-managed. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. +> - Enabled by default in GitLab 15.10. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. To help you focus on the vulnerabilities that are still relevant, GitLab SAST automatically [resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: @@ -237,7 +239,7 @@ as shown in the following table: | See new findings in merge request widget | **{dotted-circle}** | **{check-circle}** | | [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | | [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Configure SAST in the UI](#configure-sast-in-the-ui) | **{dotted-circle}** | **{check-circle}** | +| [Configure SAST by using the UI](#configure-sast-by-using-the-ui) | **{dotted-circle}** | **{check-circle}** | | [Customize SAST rulesets](customize_rulesets.md) | **{dotted-circle}** | **{check-circle}** | | [Detect False Positives](#false-positive-detection) | **{dotted-circle}** | **{check-circle}** | | [Track moved vulnerabilities](#advanced-vulnerability-tracking) | **{dotted-circle}** | **{check-circle}** | @@ -256,7 +258,7 @@ To configure SAST for a project you can: - Use [Auto SAST](../../../topics/autodevops/stages.md#auto-sast), provided by [Auto DevOps](../../../topics/autodevops/index.md). - [Configure SAST in your CI/CD YAML](#configure-sast-in-your-cicd-yaml). -- [Configure SAST using the UI](#configure-sast-in-the-ui) (introduced in GitLab 13.3). +- [Configure SAST by using the UI](#configure-sast-by-using-the-ui). You can enable SAST across many projects by [enforcing scan execution](../index.md#enforce-scan-execution). @@ -281,15 +283,12 @@ The results are saved as a that you can later download and analyze. When downloading, you always receive the most recent SAST artifact available. -### Configure SAST in the UI +### Configure SAST by using the UI -You can enable and configure SAST in the UI, either with default settings, or with customizations. +You can enable and configure SAST by using the UI, either with the default settings or with customizations. The method you can use depends on your GitLab license tier. -- [Configure SAST in the UI with customizations](#configure-sast-in-the-ui-with-customizations). **(ULTIMATE)** -- [Configure SAST in the UI with default settings only](#configure-sast-in-the-ui-with-default-settings-only). - -### Configure SAST in the UI with customizations **(ULTIMATE)** +#### Configure SAST with customizations **(ULTIMATE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab 13.3. > - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in GitLab 13.4. @@ -303,7 +302,7 @@ successfully, and an error may occur. To enable and configure SAST with customizations: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. If the project does not have a `.gitlab-ci.yml` file, select **Enable SAST** in the Static Application Security Testing (SAST) row, otherwise select **Configure SAST**. 1. Enter the custom SAST values. @@ -318,7 +317,7 @@ To enable and configure SAST with customizations: Pipelines now include a SAST job. -### Configure SAST in the UI with default settings only +#### Configure SAST with default settings only > [Introduced](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#security-configuration-page-for-all-users) in GitLab 13.9 @@ -330,7 +329,7 @@ successfully, and an error may occur. To enable and configure SAST with default settings: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance** > **Configuration**. +1. On the left sidebar, select **Security and Compliance** > **Configuration**. 1. In the SAST section, select **Configure with a merge request**. 1. Review and merge the merge request to enable SAST. @@ -600,7 +599,7 @@ Some analyzers can be customized with CI/CD variables. | `SAST_GOSEC_CONFIG` | Gosec | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328301)** in GitLab 14.0 - use custom rulesets instead. Path to configuration for Gosec (optional). | | `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | | `SAST_DISABLE_BABEL` | NodeJsScan | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64025)** in GitLab 13.5 | -| `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://r2c.dev/). Default: `true`. Introduced in GitLab 14.0 from the [confidential issue](../../project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab/-/issues/330565`. | +| `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://semgrep.dev). Default: `true`. Introduced in GitLab 14.0 from the [confidential issue](../../project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab/-/issues/330565`. | | `SAST_SCANNER_ALLOWED_CLI_OPTS` | Semgrep | CLI options (arguments with value, or flags) that are passed to the underlying security scanner when running scan operation. Only a limited set of [options](#security-scanner-configuration) are accepted. Separate a CLI option and its value using either a blank space or equals (`=`) character. For example: `name1 value1` or `name1=value1`. Multiple options must be separated by blank spaces. For example: `name1 value1 name2 value2`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368565) in GitLab 15.3. | #### Security scanner configuration @@ -617,6 +616,7 @@ flags are added to the scanner's CLI options. | Analyzer | CLI option | Description | |------------------------------------------------------------------------------|--------------------|------------------------------------------------------------------------------| | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) | `--max-memory` | Sets the maximum system memory to use when running a rule on a single file. Measured in MB. | +| [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | `--neverignore` | Never ignore security issues, even if they have an “ignore” directive in a comment. Adding this option is likely to result in the analyzer detecting additional vulnerability findings which cannot be automatically resolved. | #### Custom CI/CD variables @@ -664,7 +664,7 @@ To download the report file, you can either: - Download the file from the CI/CD pipelines page. - In the pipelines tab on merge requests, set [`artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. -For information, see [Download job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +For information, see [Download job artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts). For details of the report file's schema, see [SAST report file schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). @@ -819,7 +819,7 @@ affected. Read more in ### Getting warning message `gl-sast-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Error: `sast is used for configuration only, and its script should not be executed` diff --git a/doc/user/application_security/secret_detection/automatic_response.md b/doc/user/application_security/secret_detection/automatic_response.md new file mode 100644 index 00000000000..a898a63e33b --- /dev/null +++ b/doc/user/application_security/secret_detection/automatic_response.md @@ -0,0 +1,242 @@ +--- +stage: Secure +group: Static Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Automatic response to leaked secrets **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4639) in GitLab 13.6. + +GitLab Secret Detection automatically responds when it finds certain types of leaked secrets. +Automatic responses can: + +- Automatically revoke the secret. +- Notify the partner that issued the secret. The partner can then revoke the secret, notify its owner, or otherwise protect against abuse. + +## Supported secret types and actions + +GitLab supports automatic response for the following types of secrets: + +| Secret type | Action taken | Supported on GitLab.com | Supported in self-managed | +| ----- | --- | --- | --- | +| GitLab [Personal access tokens](../../profile/personal_access_tokens.md) | Immediately revoke token, send email to owner | ✅ | ✅ [15.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) | +| Amazon Web Services (AWS) [IAM access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) | Notify AWS | ✅ | ⚙ | + +**Component legend** + +- ✅ - Available by default +- ⚙ - Requires manual integration using a [Token Revocation API](../../../development/sec/token_revocation_api.md) + +## Feature availability + +> [Enabled for non-default branches](https://gitlab.com/gitlab-org/gitlab/-/issues/299212) in GitLab 15.11. + +Credentials are only post-processed when Secret Detection finds them: + +- In public projects, because publicly exposed credentials pose an increased threat. Expansion to private projects is considered in [issue 391379](https://gitlab.com/gitlab-org/gitlab/-/issues/391379). +- In projects with GitLab Ultimate, for technical reasons. Expansion to all tiers is tracked in [issue 391763](https://gitlab.com/gitlab-org/gitlab/-/issues/391763). + +## High-level architecture + +This diagram describes how a post-processing hook revokes a secret in the GitLab application: + +```mermaid +sequenceDiagram + autonumber + GitLab Rails-->+GitLab Rails: gl-secret-detection-report.json + GitLab Rails->>+GitLab Sidekiq: StoreScansService + GitLab Sidekiq-->+GitLab Sidekiq: ScanSecurityReportSecretsWorker + GitLab Sidekiq-->+GitLab Token Revocation API: GET revocable keys types + GitLab Token Revocation API-->>-GitLab Sidekiq: OK + GitLab Sidekiq->>+GitLab Token Revocation API: POST revoke revocable keys + GitLab Token Revocation API-->>-GitLab Sidekiq: ACCEPTED + GitLab Token Revocation API-->>+Partner API: revoke revocable keys + Partner API-->>+GitLab Token Revocation API: ACCEPTED +``` + +1. A pipeline with a Secret Detection job completes, producing a scan report (**1**). +1. The report is processed (**2**) by a service class, which schedules an asynchronous worker if token revocation is possible. +1. The asynchronous worker (**3**) communicates with an externally deployed HTTP service + (**4** and **5**) to determine which kinds of secrets can be automatically revoked. +1. The worker sends (**6** and **7**) the list of detected secrets which the GitLab Token Revocation API is able to + revoke. +1. The GitLab Token Revocation API sends (**8** and **9**) each revocable token to their respective vendor's [Partner API](#implement-a-partner-api). See the [GitLab Token Revocation API](../../../development/sec/token_revocation_api.md) + documentation for more information. + +## Partner program for leaked-credential notifications + +GitLab notifies partners when credentials they issue are leaked in public repositories on GitLab.com. +If you operate a cloud or SaaS product and you're interested in receiving these notifications, learn more in [epic 4944](https://gitlab.com/groups/gitlab-org/-/epics/4944). +Partners must [implement a Partner API](#implement-a-partner-api), which is called by the GitLab Token Revocation API. + +### Implement a Partner API + +A Partner API integrates with the GitLab Token Revocation API to receive and respond to leaked token revocation +requests. The service should be a publicly accessible HTTP API that is idempotent and rate-limited. + +Requests to your service can include one or more leaked tokens, and a header with the signature of the request +body. We strongly recommend that you verify incoming requests using this signature, to prove it's a genuine +request from GitLab. The diagram below details the necessary steps to receive, verify, and revoke leaked tokens: + +```mermaid +sequenceDiagram + autonumber + GitLab Token Revocation API-->>+Partner API: Send new leaked credentials + Partner API-->>+GitLab Public Keys endpoint: Get active public keys + GitLab Public Keys endpoint-->>+Partner API: One or more public keys + Partner API-->>+Partner API: Verify request is signed by GitLab + Partner API-->>+Partner API: Respond to leaks + Partner API-->>+GitLab Token Revocation API: HTTP status +``` + +1. The GitLab Token Revocation API sends (**1**) a [revocation request](#revocation-request) to the Partner API. The request + includes headers containing a public key identifier and signature of the request body. +1. The Partner API requests (**2**) a list of [public keys](#public-keys-endpoint) from GitLab. The response (**3**) + may include multiple public keys in the event of key rotation and should be filtered with the identifier in the request header. +1. The Partner API [verifies the signature](#verifying-the-request) against the actual request body, using the public key (**4**). +1. The Partner API processes the leaked tokens, which may involve automatic revocation (**5**). +1. The Partner API responds to the GitLab Token Revocation API (**6**) with the appropriate HTTP status code: + - A successful response code (HTTP 200 through 299) acknowledges that the partner has received and processed the request. + - An error code (HTTP 400 or higher) causes the GitLab Token Revocation API to retry the request. + +#### Revocation request + +This JSON schema document describes the body of the revocation request: + +```json +{ + "type": "array", + "items": { + "description": "A leaked token", + "type": "object", + "properties": { + "type": { + "description": "The type of token. This is vendor-specific and can be customised to suit your revocation service", + "type": "string", + "examples": [ + "my_api_token" + ] + }, + "token": { + "description": "The substring that was matched by the Secret Detection analyser. In most cases, this is the entire token itself", + "type": "string", + "examples": [ + "XXXXXXXXXXXXXXXX" + ] + }, + "url": { + "description": "The URL to the raw source file hosted on GitLab where the leaked token was detected", + "type": "string", + "examples": [ + "https://gitlab.example.com/some-repo/-/raw/abcdefghijklmnop/compromisedfile1.java" + ] + } + } + } +} +``` + +Example: + +```json +[{"type": "my_api_token", "token": "XXXXXXXXXXXXXXXX", "url": "https://example.com/some-repo/-/raw/abcdefghijklmnop/compromisedfile1.java"}] +``` + +In this example, Secret Detection has determined that an instance of `my_api_token` has been leaked. The +value of the token is provided to you, in addition to a publicly accessible URL to the raw content of the +file containing the leaked token. + +The request includes two special headers: + +| Header | Type | Description | +|--------|------|-------------| +| `Gitlab-Public-Key-Identifier` | string | A unique identifier for the key pair used to sign this request. Primarily used to aid in key rotation. | +| `Gitlab-Public-Key-Signature` | string | A base64-encoded signature of the request body. | + +You can use these headers along with the GitLab Public Keys endpoint to verify that the revocation request was genuine. + +#### Public Keys endpoint + +GitLab maintains a publicly-accessible endpoint for retrieving public keys used to verify revocation +requests. The endpoint can be provided on request. + +This JSON schema document describes the response body of the public keys endpoint: + +```json +{ + "type": "object", + "properties": { + "public_keys": { + "description": "An array of public keys managed by GitLab used to sign token revocation requests.", + "type": "array", + "items": { + "type": "object", + "properties": { + "key_identifier": { + "description": "A unique identifier for the keypair. Match this against the value of the Gitlab-Public-Key-Identifier header", + "type": "string" + }, + "key": { + "description": "The value of the public key", + "type": "string" + }, + "is_current": { + "description": "Whether the key is currently active and signing new requests", + "type": "boolean" + } + } + } + } + } +} +``` + +Example: + +```json +{ + "public_keys": [ + { + "key_identifier": "6917d7584f0fa65c8c33df5ab20f54dfb9a6e6ae", + "key": "-----BEGIN PUBLIC KEY-----\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEN05/VjsBwWTUGYMpijqC5pDtoLEf\nuWz2CVZAZd5zfa/NAlSFgWRDdNRpazTARndB2+dHDtcHIVfzyVPNr2aznw==\n-----END PUBLIC KEY-----\n", + "is_current": true + } + ] +} +``` + +#### Verifying the request + +You can check whether a revocation request is genuine by verifying the `Gitlab-Public-Key-Signature` header +against the request body, using the corresponding public key taken from the API response above. We use +[ECDSA](https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm) with SHA256 hashing to +produce the signature, which is then base64-encoded into the header value. + +The Python script below demonstrates how the signature can be verified. It uses the popular +[pyca/cryptography](https://cryptography.io/en/latest/) module for cryptographic operations: + +```python +import hashlib +import base64 +from cryptography.hazmat.primitives import hashes +from cryptography.hazmat.primitives.serialization import load_pem_public_key +from cryptography.hazmat.primitives.asymmetric import ec + +public_key = str.encode("") # obtained from the public keys endpoint +signature_header = "" # obtained from the `Gitlab-Public-Key-Signature` header +request_body = str.encode(r'') # obtained from the revocation request body + +pk = load_pem_public_key(public_key) +decoded_signature = base64.b64decode(signature_header) + +pk.verify(decoded_signature, request_body, ec.ECDSA(hashes.SHA256())) # throws if unsuccessful + +print("Signature verified!") +``` + +The main steps are: + +1. Loading the public key into a format appropriate for the crypto library you're using. +1. Base64-decoding the `Gitlab-Public-Key-Signature` header value. +1. Verifying the body against the decoded signature, specifying ECDSA with SHA256 hashing. diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index d6aab71a2c6..5a1dcc840ca 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -15,40 +15,42 @@ info: To determine the technical writer assigned to the Stage/Group associated w > `secret_detection_default_branch` and `secret_detection` were consolidated into one job, > `secret_detection`. -People may accidentally commit secrets (such as keys, passwords, and API tokens) to remote Git repositories. +People sometimes accidentally commit secrets like keys or API tokens to Git repositories. +After a sensitive value is pushed to a remote repository, anyone with access to the repository can impersonate the authorized user of the secret for malicious purposes. +Most organizations require exposed secrets to be revoked and replaced to address this risk. -Anyone with access to the repository could use the secrets for malicious purposes. Exposed secrets -must be considered compromised and be replaced, which can be costly. +Secret Detection scans your repository to help prevent your secrets from being exposed. +Secret Detection scanning works on all text files, regardless of the language or framework used. -To help prevent secrets from being committed to a Git repository, you can use Secret Detection to -scan your repository for secrets. Scanning is language and framework agnostic, but does not support -scanning binary files. +After you [enable Secret Detection](#enable-secret-detection), scans run in a CI/CD job named `secret_detection`. +You can run scans and view [Secret Detection JSON report artifacts](../../../ci/yaml/artifacts_reports.md#artifactsreportssecret_detection) in any GitLab tier. -Secret Detection uses an analyzer containing the [Gitleaks](https://github.com/zricethezav/gitleaks) -tool to scan the repository for secrets. Detection occurs in the `secret-detection` job. The results -are saved as a -[Secret Detection report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportssecret_detection) -that you can later download and analyze. Due to implementation limitations, we always take the -latest Secret Detection artifact available. +With GitLab Ultimate, Secret Detection results are also processed so you can: -GitLab SaaS supports post-processing hooks, so you can take action when a secret is found. For -more information, see [Post-processing and revocation](post_processing.md). +- See them in the [merge request widget](../index.md#view-security-scan-information-in-merge-requests), [pipeline security report](../vulnerability_report/pipeline.md), and [Vulnerability Report](../vulnerability_report/index.md). +- Use them in approval workflows. +- Review them in the security dashboard. +- [Automatically respond](automatic_response.md) to leaks in public repositories. -All identified secrets are reported in the: +## Detected secrets -- Merge request widget -- Pipelines' **Security** tab -- [Vulnerability Report](../vulnerability_report/index.md) +GitLab maintains the detection rules used in Secret Detection. +The [default ruleset](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/master/gitleaks.toml) +contains more than 100 patterns. -![Secret Detection in merge request widget](img/secret_detection_v13_2.png) +Most Secret Detection patterns search for specific types of secrets. +Many services add prefixes or other structural details to their secrets so they can be identified if they're leaked. +For example, GitLab [adds a `glpat-` prefix](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) to project, group, and personal access tokens by default. -## Detected secrets +To provide more reliable, high-confidence results, Secret Detection only looks for passwords or other unstructured secrets in specific contexts like URLs. + +### Adding new patterns + +To search for other types of secrets in your repositories, you can configure a [custom ruleset](#custom-rulesets). -Secret Detection uses a [default ruleset](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/master/gitleaks.toml) -containing more than 90 secret detection patterns. You can also customize the secret detection -patterns using [custom rulesets](#custom-rulesets). If you want to contribute rulesets for -"well-identifiable" secrets, follow the steps detailed in the -[community contributions guidelines](https://gitlab.com/gitlab-org/gitlab/-/issues/345453). +To propose a new detection rule for all users of Secret Detection, create a merge request against the [file containing the default rules](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/master/gitleaks.toml). + +If you operate a cloud or SaaS product and you're interested in partnering with GitLab to better protect your users, learn more about our [partner program for leaked credential notifications](automatic_response.md#partner-program-for-leaked-credential-notifications). ## Features per tier @@ -59,6 +61,7 @@ Different features are available in different [GitLab tiers](https://about.gitla | [Configure Secret Detection scanner](#enable-secret-detection) | **{check-circle}** Yes | **{check-circle}** Yes | | [Customize Secret Detection settings](#configure-scan-settings) | **{check-circle}** Yes | **{check-circle}** Yes | | Download [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** Yes | **{check-circle}** Yes | +| [Check text for potential secrets](#warnings-for-potential-leaks-in-text-content) before it's posted | **{check-circle}** Yes | **{check-circle}** Yes | | See new findings in the merge request widget | **{dotted-circle}** No | **{check-circle}** Yes | | View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** No | **{check-circle}** Yes | | [Manage vulnerabilities](../vulnerability_report/index.md) | **{dotted-circle}** No | **{check-circle}** Yes | @@ -141,12 +144,12 @@ To enable Secret Detection, either: - Enable [Auto DevOps](../../../topics/autodevops/index.md), which includes [Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection). -- [Edit the `.gitlab.ci.yml` file manually](#edit-the-gitlabciyml-file-manually). Use this method if - your `.gitlab-ci.yml` file is complex. +- [Edit the `.gitlab-ci.yml` file manually](#edit-the-gitlab-ciyml-file-manually). Use this method + if your `.gitlab-ci.yml` file is complex. - [Use an automatically configured merge request](#use-an-automatically-configured-merge-request). -### Edit the `.gitlab.ci.yml` file manually +### Edit the `.gitlab-ci.yml` file manually This method requires you to manually edit the existing `.gitlab-ci.yml` file. Use this method if your GitLab CI/CD configuration file is complex. @@ -180,12 +183,12 @@ the `.gitlab-ci.yml` file. You then merge the merge request to enable Secret Det NOTE: This method works best with no existing `.gitlab-ci.yml` file, or with a minimal configuration file. If you have a complex GitLab configuration file it may not be parsed successfully, and an -error may occur. In that case, use the [manual](#edit-the-gitlabciyml-file-manually) method instead. +error may occur. In that case, use the [manual](#edit-the-gitlab-ciyml-file-manually) method instead. -To enable Secret Detection automatically: +To enable Secret Detection: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. In the **Secret Detection** row, select **Configure with a merge request**. 1. Optional. Complete the fields. 1. Select **Create merge request**. @@ -195,12 +198,13 @@ Pipelines now include a Secret Detection job. ## Responding to a leaked secret -Secrets detected by the analyzer should be immediately rotated. -[Purging a file from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history) -may not be effective in removing all references to the file. Additionally, the secret will remain in any existing -forks or clones of the repository. +When a secret is detected, you should rotate it immediately. GitLab attempts to +[automatically revoke](automatic_response.md) some types of leaked secrets. For those that are not +automatically revoked, you must do so manually. -GitLab will attempt to [automatically revoke](post_processing.md) some types of leaked secrets. +[Purging a secret from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history) +does not fully address the leak. The original secret remains in any existing forks or +clones of the repository. ## Pinning to specific analyzer version @@ -413,18 +417,16 @@ In the following example `secret-detection-ruleset.toml` file, rules are matched ### Synthesize a custom configuration -To create a custom configuration, you can use passthrough chains. Passthroughs can also be chained -to build more complex configurations. For more details, see -[SAST Customize ruleset](../sast/customize_rulesets.md). - -Only the following passthrough types are supported by the `secrets` analyzer: +You can use passthroughs to override the default Secret Detection ruleset. The +following passthrough types are supported by the `secrets` analyzer: -- `file` - `raw` +- `file` -In the `secret-detection-ruleset.toml` file, do one of the following: +To define a passthrough, add _one_ of the following to the +`secret-detection-ruleset.toml` file: -- Define a custom ruleset, for example: +- Using an inline (`raw`) value: ```toml [secrets] @@ -442,7 +444,7 @@ In the `secret-detection-ruleset.toml` file, do one of the following: """ ``` -- Provide the name of the file containing a custom ruleset, for example: +- Using an external `file` committed to the current repository: ```toml [secrets] @@ -454,6 +456,45 @@ In the `secret-detection-ruleset.toml` file, do one of the following: value = "config/gitleaks.toml" ``` +For more information on the syntax of passthroughs, see the +[passthroughs section on the SAST customize rulesets](../sast/customize_rulesets.md#the-analyzerpassthrough-section) +page. + +#### Extending the default configuration + +You can extend the default configuration with additional changes by using [Gitleaks `extend` support](https://github.com/gitleaks/gitleaks#configuration). + +In the following `file` passthrough example, the string `glpat-1234567890abcdefghij` is ignored by Secret Detection. That GitLab personal access token (PAT) is used in test cases. Detection of it would be a false positive. + +The `secret-detection-ruleset.toml` file defines that the configuration in `extended-gitleaks-config.toml` file is to be included. The `extended-gitleaks-config.toml` file defines the custom Gitleaks configuration. The `allowlist` stanza defines a regular expression that matches the secret that is to be ignored ("allowed"). + +```toml +# .gitlab/secret-detection-ruleset.toml +[secrets] + description = 'secrets custom rules configuration' + + [[secrets.passthrough]] + type = "file" + target = "gitleaks.toml" + value = "extended-gitleaks-config.toml" +``` + +```toml +# extended-gitleaks-config.toml +title = "extension of gitlab's default gitleaks config" + +[extend] +# Extends default packaged path +path = "/gitleaks.toml" + +[allowlist] + description = "allow list of test tokens to ignore in detection" + regexTarget = "match" + regexes = [ + '''glpat-1234567890abcdefghij''', + ] +``` + ## Running Secret Detection in an offline environment **(PREMIUM SELF)** An offline environment has limited, restricted, or intermittent access to external resources through @@ -532,6 +573,26 @@ variable, or as a CI/CD variable. - If using a variable, set the value of `ADDITIONAL_CA_CERT_BUNDLE` to the text representation of the certificate. +## Warnings for potential leaks in text content + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368434) in GitLab 15.11. + +When you create an issue, propose a merge request, or write a comment, you might accidentally post a sensitive value. +For example, you might paste in the details of an API request or an environment variable that contains an authentication token. + +GitLab checks if the text of your issue description, merge request description, comment, or reply contains a sensitive token. +If a token is found, a warning message is displayed. You can then edit your message before posting it. +This check happens in your browser before the message is sent to the server. +The check is always on; you don't have to set it up. + +Your text is checked for the following secret types: + +- GitLab [personal access tokens](../../../security/token_overview.md#personal-access-tokens) +- GitLab [feed tokens](../../../security/token_overview.md#feed-token) + +This feature is separate from Secret Detection scanning, which checks your Git repository for leaked secrets. +[Issue 405147](https://gitlab.com/gitlab-org/gitlab/-/issues/405147) tracks efforts to align these two types of protection. + ## Troubleshooting ### Set the logging level @@ -552,7 +613,7 @@ visible in job logs. ### Warning: `gl-secret-detection-report.json: no matching files` -For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). +For information on this, see the [general Application Security troubleshooting section](../../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload). ### Error: `Couldn't run the gitleaks command: exit status 2` diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md index 22d7a8ba5af..3a6cf7f7e37 100644 --- a/doc/user/application_security/secret_detection/post_processing.md +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -1,97 +1,11 @@ --- -stage: Secure -group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'automatic_response.md' +remove_date: '2023-08-08' --- -# Secret Detection post-processing and revocation **(ULTIMATE SAAS)** +This document was moved to [another location](automatic_response.md). -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4639) in GitLab 13.6. -> - [Disabled by default for GitLab personal access tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `gitlab_pat_auto_revocation`. Available to GitLab.com only. -> - [Enabled by default for GitLab personal access tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) in GitLab 15.9 - -GitLab.com and self-managed supports running post-processing hooks after detecting a secret. These -hooks can perform actions, like notifying the vendor that issued the secret. -The vendor can then confirm the credentials and take remediation actions, like: - -- Revoking a secret. -- Reissuing a secret. -- Notifying the creator of the secret. - -GitLab supports post-processing for the following vendors and secrets: - -| Vendor | Secret | GitLab.com | Self-managed | -| ----- | --- | --- | --- | -| GitLab | [Personal access tokens](../../profile/personal_access_tokens.md) | ✅ | ✅ 15.9 and later | -| Amazon Web Services (AWS) | [IAM access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) | ✅ | ⚙ | - -**Component legend** - -- ✅ - Available by default -- ⚙ - Requires manual integration using a [Token Revocation API](../../../development/sec/token_revocation_api.md) - -## Feature availability - -Credentials are only post-processed when Secret Detection finds them: - -- In public projects, because publicly exposed credentials pose an increased threat. Expansion to private projects is considered in [issue 391379](https://gitlab.com/gitlab-org/gitlab/-/issues/391379). -- On the project [default branch](../../project/repository/branches/default.md), for technical reasons. Expansion to all branches is tracked in [issue 299212](https://gitlab.com/gitlab-org/gitlab/-/issues/299212). -- In projects with GitLab Ultimate, for technical reasons. Expansion to all tiers is tracked in [issue 391763](https://gitlab.com/gitlab-org/gitlab/-/issues/391763). - -## High-level architecture - -This diagram describes how a post-processing hook revokes a secret within the GitLab application: - -```mermaid -sequenceDiagram - autonumber - GitLab Rails->>+Sidekiq: gl-secret-detection-report.json - Sidekiq-->+Sidekiq: StoreSecurityReportsWorker - Sidekiq-->+Token Revocation API: GET revocable keys types - Token Revocation API-->>-Sidekiq: OK - Sidekiq->>+Token Revocation API: POST revoke revocable keys - Token Revocation API-->>-Sidekiq: ACCEPTED - Token Revocation API-->>+Receiver Service: revoke revocable keys - Receiver Service-->>+Token Revocation API: ACCEPTED -``` - -1. A pipeline with a Secret Detection job completes on the project's default branch, producing a scan - report (**1**). -1. The report is processed (**2**) by an asynchronous worker, which communicates with an externally - deployed HTTP service (**3** and **4**) to determine which kinds of secrets can be automatically - revoked. -1. The worker sends (**5** and **6**) the list of detected secrets which the Token Revocation API is able to - revoke. -1. The Token Revocation API sends (**7** and **8**) each revocable token to their respective vendor's [receiver service](#integrate-your-cloud-provider-service-with-gitlabcom). - -See the [Token Revocation API](../../../development/sec/token_revocation_api.md) documentation for more -information. - -## Integrate your cloud provider service with GitLab.com - -Third-party cloud and SaaS vendors interested in automated token revocation can -[express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). -Vendors must [implement a revocation receiver service](#implement-a-revocation-receiver-service) -which will be called by the Token Revocation API. - -### Implement a revocation receiver service - -A revocation receiver service integrates with a GitLab instance's Token Revocation API to receive and respond -to leaked token revocation requests. The service should be a publicly accessible HTTP API that is -idempotent and rate-limited. Requests to your service from the Token Revocation API will follow the example -below: - -```plaintext -POST / HTTP/2 -Accept: */* -Content-Type: application/json -X-Gitlab-Token: MYSECRETTOKEN - -[ - {"type": "my_api_token", "token":"XXXXXXXXXXXXXXXX","url": "https://example.com/some-repo/~/raw/abcdefghijklmnop/compromisedfile1.java"} -] -``` - -In this example, Secret Detection has determined that an instance of `my_api_token` has been leaked. The -value of the token is provided to you, in addition to a publicly accessible URL to the raw content of the -file containing the leaked token. +<!-- This redirect file can be deleted after 2023-08-08. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/application_security/secure_your_application.md b/doc/user/application_security/secure_your_application.md index fb10efff2c6..8dd1168a0d9 100644 --- a/doc/user/application_security/secure_your_application.md +++ b/doc/user/application_security/secure_your_application.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab can check your applications for security vulnerabilities. - [Get started](get-started-security.md) -- [Security Configuration](configuration/index.md) +- [Security configuration](configuration/index.md) - [Container Scanning](container_scanning/index.md) - [Dependency Scanning](dependency_scanning/index.md) - [Static Application Security Testing](sast/index.md) diff --git a/doc/user/application_security/security_dashboard/img/security_center_dashboard_v13_4.png b/doc/user/application_security/security_dashboard/img/security_center_dashboard_v13_4.png Binary files differdeleted file mode 100644 index 5379b5c6e5d..00000000000 --- a/doc/user/application_security/security_dashboard/img/security_center_dashboard_v13_4.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png b/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png Binary files differnew file mode 100644 index 00000000000..c2780fce787 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/security_center_dashboard_v15_10.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 7388ef80e68..e6cb4cef4b2 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -19,6 +19,9 @@ To use the Security Dashboards, you must: shared runners on GitLab.com, you are using the correct version. - Have the [correct role](../../permissions.md) for the project or group. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Security Dashboard](https://www.youtube.com/watch?v=QHQHN4luNpc). + ## When Security Dashboards are updated The Security Dashboards show results of scans from the most recent completed pipeline on the @@ -64,7 +67,7 @@ Project Security Dashboards show statistics for all vulnerabilities with a curre To view total number of vulnerabilities over time: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Security Dashboard**. +1. On the left sidebar, select **Security and Compliance > Security Dashboard**. 1. Filter and search for what you need. - To filter the chart by severity, select the legend name. - To view a specific time frame, use the time range handles (**{scroll-handle}**). @@ -77,7 +80,7 @@ To view total number of vulnerabilities over time: To download an SVG image of the vulnerabilities chart: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Security dashboard**. +1. On the left sidebar, select **Security and Compliance > Security dashboard**. 1. Select **Save chart as an image** (**{download}**). ## View vulnerabilities over time for a group @@ -133,7 +136,7 @@ The Security Center includes: - A [vulnerability report](../vulnerability_report/index.md). - A settings area to configure which projects to display. -![Security Center Dashboard with projects](img/security_center_dashboard_v13_4.png) +![Security Center Dashboard with projects](img/security_center_dashboard_v15_10.png) ### View the Security Center @@ -151,6 +154,9 @@ To add projects to the Security Center: After you add projects, the security dashboard and vulnerability report show the vulnerabilities found in those projects' default branches. +You can add a maximum of 1,000 projects, however the **Project** filter in the **Vulnerability +Report** is limited to 100 projects. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index 772a7d17e1e..df103976901 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Secure and Govern terminology **(FREE)** +# Secure and Govern glossary **(FREE)** The glossary of terms aims to achieve the following: @@ -17,9 +17,7 @@ The glossary of terms aims to achieve the following: The definitions of the terms outlined in this document are in the context of the GitLab products. Therefore, a term may have a different meaning to users outside of GitLab. -## Terms - -### Analyzer +## Analyzer Software that performs a scan. The scan analyzes an attack surface for vulnerabilities and produces a report containing findings. Reports adhere to the [Secure report format](#secure-report-format). @@ -31,35 +29,35 @@ manage found vulnerabilities. For more information, see [Security Scanner Integr Many GitLab analyzers follow a standard approach using Docker to run a wrapped scanner. For example, the image `semgrep` is an analyzer that wraps the scanner `Semgrep`. -### Attack surface +## Attack surface The different places in an application that are vulnerable to attack. Secure products discover and search the attack surface during scans. Each product defines the attack surface differently. For example, SAST uses files and line numbers, and DAST uses URLs. -### Corpus +## Corpus The set of meaningful test cases that are generated while the fuzzer is running. Each meaningful test case produces new coverage in the tested program. It's advised to re-use the corpus and pass it to subsequent runs. -### CNA +## CNA [CVE](#cve) Numbering Authorities (CNAs) are organizations from around the world that are authorized by the [Mitre Corporation](https://cve.mitre.org/) to assign [CVE](#cve)s to vulnerabilities in products or services within their respective scope. [GitLab is a CNA](https://about.gitlab.com/security/cve/). -### CVE +## CVE Common Vulnerabilities and Exposures (CVE®) is a list of common identifiers for publicly known cybersecurity vulnerabilities. The list is managed by the [Mitre Corporation](https://cve.mitre.org/). -### CVSS +## CVSS The Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing the severity of computer system security vulnerabilities. -### CWE +## CWE Common Weakness Enumeration (CWE™) is a community-developed list of common software and hardware weakness types that have security ramifications. Weaknesses are flaws, faults, bugs, @@ -68,22 +66,22 @@ architecture. If left unaddressed, weaknesses could result in systems, networks, vulnerable to attack. The CWE List and associated classification taxonomy serve as a language that you can use to identify and describe these weaknesses in terms of CWEs. -### Deduplication +## Deduplication When a category's process deems findings to be the same, or if they are similar enough that a noise reduction is required, only one finding is kept and the others are eliminated. Read more about the [deduplication process](../vulnerability_report/pipeline.md#deduplication-process). -### Duplicate finding +## Duplicate finding A legitimate finding that is reported multiple times. This can occur when different scanners discover the same finding, or when a single scan inadvertently reports the same finding more than once. -### False positive +## False positive A finding that doesn't exist but is incorrectly reported as existing. -### Finding +## Finding An asset that has the potential to be vulnerable, identified in a project by an analyzer. Assets include but are not restricted to source code, binary packages, containers, dependencies, networks, @@ -96,32 +94,32 @@ You can interact with vulnerability findings in two ways. 1. You can open an issue or merge request for the vulnerability finding. 1. You can dismiss the vulnerability finding. Dismissing the finding hides it from the default views. -### Grouping +## Grouping A flexible and non-destructive way to visually organize vulnerabilities in groups when there are multiple findings that are likely related but do not qualify for deduplication. For example, you can include findings that should be evaluated together, would be fixed by the same action, or come from the same source. Grouping behavior for vulnerabilities is under development and tracked in issue [267588](https://gitlab.com/gitlab-org/gitlab/-/issues/267588). -### Insignificant finding +## Insignificant finding A legitimate finding that a particular customer doesn't care about. -### Location fingerprint +## Location fingerprint A finding's location fingerprint is a text value that's unique for each location on the attack surface. Each security product defines this according to its type of attack surface. For example, SAST incorporates file path and line number. -### Package managers and package types +## Package managers and package types -#### Package managers +### Package managers A package manager is a system that manages your project dependencies. The package manager provides a method to install new dependencies (also referred to as "packages"), manage where packages are stored on your file system, and offer capabilities for you to publish your own packages. -#### Package types +### Package types Each package manager, platform, type, or ecosystem has its own conventions and protocols to identify, locate, and provision software packages. @@ -215,11 +213,11 @@ table.package-managers-and-types ul { </tbody> </table> -### Pipeline Security tab +## Pipeline Security tab A page that displays findings discovered in the associated CI pipeline. -### Post-filter +## Post-filter Post-filters help reduce noise in the scanner results and automate manual tasks. You can specify criteria that updates or modifies vulnerability data based on scanner results. For example, you can flag findings as likely False Positives @@ -228,7 +226,7 @@ and automatically resolve vulnerabilities that are no longer detected. These are Support for automatically resolving findings is tracked in epic [7478](https://gitlab.com/groups/gitlab-org/-/epics/7478) and support for cheap scan is proposed in issue [349926](https://gitlab.com/gitlab-org/gitlab/-/issues/349926). -### Pre-filter +## Pre-filter An irreversible action that is done to filter out targets before analysis occurs. This is usually provided to allow the user to reduce scope and noise as well as speed up the analysis. This should not be done if a record is needed as @@ -236,7 +234,7 @@ we currently do not store anything related to the skipped/excluded code or asset Examples: `DS_EXCLUDED_PATHS` should `Exclude files and directories from the scan based on the paths provided.` -### Primary identifier +## Primary identifier A finding's primary identifier is a value that is unique to each finding. The external type and external ID of the finding's [first identifier](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/v2.4.0-rc1/dist/sast-report-format.json#L228) @@ -246,13 +244,13 @@ Examples of primary identifiers include `PluginID` for OWASP Zed Attack Proxy (Z Trivy. The identifier must be stable. Subsequent scans must return the same value for the same finding, even if the location has slightly changed. -### Report finding +## Report finding A [finding](#finding) that only exists in a report produced by an analyzer, and is yet to be persisted to the database. The report finding becomes a [vulnerability finding](#vulnerability-finding) once it's imported into the database. -### Scan type (report type) +## Scan type (report type) Describes the type of scan. This must be one of the following: @@ -266,12 +264,12 @@ Describes the type of scan. This must be one of the following: This list is subject to change as scanners are added. -### Scanner +## Scanner Software that can scan for vulnerabilities. The resulting scan report is typically not in the [Secure report format](#secure-report-format). Examples include ESLint, Trivy, and ZAP. -### Secure product +## Secure product A group of features related to a specific area of application security with first-class support by GitLab. @@ -281,23 +279,23 @@ Testing (DAST), Secret Detection, Static Application Security Testing (SAST), an Each of these products typically include one or more analyzers. -### Secure report format +## Secure report format A standard report format that Secure products comply with when creating JSON reports. The format is described by a [JSON schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas). -### Security Dashboard +## Security Dashboard Provides an overview of all the vulnerabilities for a project, group, or GitLab instance. Vulnerabilities are only created from findings discovered on the project's default branch. -### Seed corpus +## Seed corpus The set of test cases given as initial input to the fuzz target. This usually speeds up the fuzz target substantially. This can be either manually created test cases or auto-generated with the fuzz target itself from previous runs. -### Vendor +## Vendor The party maintaining an analyzer. As such, a vendor is responsible for integrating a scanner into GitLab and keeping it compatible as they evolve. A vendor isn't necessarily the author or maintainer @@ -305,7 +303,7 @@ of the scanner, as in the case of using an open core or OSS project as a base so offering. For scanners included as part of a GitLab distribution or GitLab subscription, the vendor is listed as GitLab. -### Vulnerability +## Vulnerability A flaw that has a negative impact on the security of its environment. Vulnerabilities describe the error or weakness, and don't describe where the error is located (see [finding](#finding)). @@ -314,12 +312,12 @@ Each vulnerability maps to a unique finding. Vulnerabilities exist in the default branch. Findings (see [finding](#finding)) are all potential vulnerability items scanners identify in MRs/feature branches. Only after merging to default does a finding become a vulnerability. -### Vulnerability finding +## Vulnerability finding When a [report finding](#report-finding) is stored to the database, it becomes a vulnerability [finding](#finding). -### Vulnerability tracking +## Vulnerability tracking Deals with the responsibility of matching findings across scans so that a finding's life cycle can be understood. Engineers and security teams use this information to decide whether to merge code @@ -327,6 +325,6 @@ changes, and to see unresolved findings and when they were introduced. Vulnerabilities are tracked by comparing the location fingerprint, primary identifier, and report type. -### Vulnerability occurrence +## Vulnerability occurrence Deprecated, see [finding](#finding). diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 67a1257799b..18485f83fe7 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -8,10 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13561) in GitLab 13.0. -Each vulnerability in a project has a Vulnerability Page, containing details of the -vulnerability. The details included vary according to the type of vulnerability. - -Details of each vulnerability include: +Each vulnerability in a project has a vulnerability page containing details of the vulnerability, +including: - Description - When it was detected @@ -20,17 +18,13 @@ Details of each vulnerability include: - Linked issues - Actions log -In GitLab 14.3 and later, if the scanner determined the vulnerability to be a false positive, an -alert message is included at the top of the vulnerability's page. - -On the vulnerability's page, you can: +If the scanner determined the vulnerability to be a false positive, an alert message is included at +the top of the vulnerability's page. -- [Change the vulnerability's status](#change-status-of-a-vulnerability). -- [Create an issue](#creating-an-issue-for-a-vulnerability). -- [Link issues to the vulnerability](#linking-a-vulnerability-to-issues). -- [Resolve the vulnerability](#resolve-a-vulnerability) if a solution is - available. -- [View security training specific to the detected vulnerability](#view-security-training-for-a-vulnerability). +When a vulnerability is no longer detected in a project's default branch, you should +change its status to **Resolved**. This ensures that if it is accidentally reintroduced in a future +merge, it is reported again as a new record. To change the status of multiple vulnerabilities, use +the Vulnerability Report's [Activity filter](../vulnerability_report/index.md#activity-filter). ## Vulnerability status values @@ -38,28 +32,41 @@ A vulnerability's status can be: - **Detected**: The default state for a newly discovered vulnerability. Appears as "Needs triage" in the UI. - **Confirmed**: A user has seen this vulnerability and confirmed it to be accurate. -- **Dismissed**: A user has seen this vulnerability and dismissed it because it is not accurate or otherwise not to be resolved. -- **Resolved**: The vulnerability has been fixed or is no longer present. +- **Dismissed**: A user has seen this vulnerability and dismissed it because it is not accurate or + otherwise not to be resolved. Dismissed vulnerabilities are ignored if detected in subsequent + scans. +- **Resolved**: The vulnerability has been fixed or is no longer present. Resolved vulnerabilities + that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. + +## Vulnerability dismissal reasons + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4942) in GitLab 15.11 with a feature flag named `dismissal_reason`. +> - Enabled on GitLab.com in GitLab 15.11. For self-managed customers, [contact Support](https://about.gitlab.com/support/) if you would like to use this feature in GitLab 15.11. +> - Enabled by default in GitLab 16.0. + +When dismissing a vulnerability, one of the following reasons must be chosen to clarify why it is being dismissed: -Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that -are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an -existing vulnerability is no longer detected in a project's `default` branch, you should change its -status to **Resolved**. This ensures that if it is accidentally reintroduced in a future merge, it -is reported again as a new record. You can use the Vulnerability Report's -[Activity filter](../vulnerability_report/index.md#activity-filter) to select all vulnerabilities that are -no longer detected, and change their status. +- **Acceptable risk**: The vulnerability is known, and has not been remediated or mitigated, but is considered to be an acceptable business risk. +- **False positive**: An error in reporting in which a test result incorrectly indicates the presence of a vulnerability in a system when the vulnerability is not present. +- **Mitigating control**: A management, operational, or technical control (that is, safeguard or countermeasure) employed by an organization that provides equivalent or comparable protection for an information system. +- **Used in tests**: The finding is not a vulnerability because it is part of a test or is test data. +- **Not applicable**: The vulnerability is known, and has not been remediated or mitigated, but is considered to be in a part of the application that will not be updated. ## Change status of a vulnerability To change a vulnerability's status from its Vulnerability Page: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Status** dropdown list select a status, then select **Change status**. + + In GitLab 15.11 and later, you must select a [dismissal reason](#vulnerability-dismissal-reasons) when you change a vulnerability's status to **Dismissed**. + 1. Optionally, at the bottom of the page, add a comment to the log entry. -The Actions log records each status change along with which user changed the status and the time of the change. +Details of the status change, including who made the change and when, are recorded in the +vulnerability's action log. ## Creating an issue for a vulnerability @@ -80,7 +87,7 @@ that when Jira integration is enabled, the GitLab issue feature is not available To create a GitLab issue for a vulnerability: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. Select **Create issue**. @@ -102,7 +109,7 @@ Prerequisites: To create a Jira issue for a vulnerability: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. Select **Create Jira issue**. 1. If you're not already logged in to Jira, sign in. @@ -135,7 +142,7 @@ Be aware of the following conditions between a vulnerability and a linked issue: To link a vulnerability to existing issues: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. In the **Linked issues** section, select the plus icon (**{plus}**). 1. For each issue to be linked, either: @@ -170,7 +177,7 @@ To resolve a vulnerability, you can either: To resolve the vulnerability with a merge request: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Resolve with merge request** dropdown list, select **Resolve with merge request**. @@ -182,7 +189,7 @@ Process the merge request according to your standard workflow. To manually apply the patch that GitLab generated for a vulnerability: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability's description. 1. From the **Resolve with merge request** dropdown list, select **Download patch to resolve**. 1. Ensure your local project has the same commit checked out that was used to generate the patch. @@ -195,17 +202,19 @@ To manually apply the patch that GitLab generated for a vulnerability: > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. +NOTE: +Security training is not available in an offline environment because it uses content from +third-party vendors. + Security training helps your developers learn how to fix vulnerabilities. Developers can view security training from selected educational providers, relevant to the detected vulnerability. To enable security training for vulnerabilities in your project: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the left sidebar, select **Security and Compliance > Security configuration**. 1. On the tab bar, select **Vulnerability Management**. 1. To enable a security training provider, turn on the toggle. -Security training uses content from third-party vendors. You must have an internet connection to use this feature. - ## View security training for a vulnerability > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. @@ -220,6 +229,6 @@ Vulnerabilities with a CWE are most likely to return a training result. To view the security training for a vulnerability: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. 1. Select the vulnerability for which you want to view security training. 1. Select **View training**. diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index e75d0a45f7d..ab90ac18b8e 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -56,7 +56,7 @@ the following tables: |------------------------------------------------------------------------------------------|------------------------------|----------------------------|-------------------------------------| | [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | **{check-circle}** Yes | CVSS v2.0 Rating and CVSS v3.1 Qualitative Severity Rating <sup>1</sup> | `(AV:N/AC:L/Au:S/C:P/I:P/A:N)`, `CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H` | -1. The CVSS v3.1 rating is used to calculate the severity level. If it's not available, the CVSS v2.0 rating is used instead. +The CVSS v3.1 rating is used to calculate the severity level. If it's not available, the CVSS v2.0 rating is used instead. ## Container Scanning @@ -64,6 +64,8 @@ the following tables: |------------------------------------------------------------------------|--------------------------|----------------------------|--------------------------------------------------------------| | [`container-scanning`](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning)| **{check-circle}** Yes | String | `Unknown`, `Low`, `Medium`, `High`, `Critical` | +When available, the vendor severity level takes precedence and is used by the analyzer. If that is not available then it falls back on the CVSS v3.1 rating. If that is also not available, then the CVSS v2.0 rating is used instead. Details on this implementation are available on the respective issues for [trivy](https://github.com/aquasecurity/trivy/issues/310) and [grype](https://github.com/anchore/grype/issues/287). + ## Fuzz Testing All fuzz testing results are reported as Unknown. They should be reviewed and triaged manually to find exploitable faults to prioritize for fixing. diff --git a/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v14_2.png b/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v14_2.png Binary files differdeleted file mode 100644 index a43340544ca..00000000000 --- a/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v14_2.png +++ /dev/null diff --git a/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v16_0.png b/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v16_0.png Binary files differnew file mode 100644 index 00000000000..fd9626be2d9 --- /dev/null +++ b/doc/user/application_security/vulnerability_report/img/project_security_dashboard_status_change_v16_0.png diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index e6353264f39..0826258de9e 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -19,6 +19,9 @@ At all levels, the Vulnerability Report contains: - Filters for common vulnerability attributes. - Details of each vulnerability, presented in tabular layout. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Vulnerability Management](https://www.youtube.com/watch?v=8SJHz6BCgXM). + The **Activity** column contains icons to indicate the activity, if any, taken on the vulnerability in that row: @@ -45,7 +48,7 @@ At the project level, the Vulnerability Report also contains: To view the project-level vulnerability report: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability report**. ## Vulnerability Report actions @@ -158,7 +161,8 @@ If Jira issue support is enabled, the issue link found in the Activity entry lin ## Change status of vulnerabilities -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292636) in GitLab 13.10, all statuses became selectable. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292636) in GitLab 13.10, all statuses became selectable. +> - Providing a comment and dismissal reason [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408366) in GitLab 16.0. From the Vulnerability Report you can change the status of one or more vulnerabilities. @@ -167,9 +171,11 @@ To change the status of vulnerabilities in the table: 1. Select the checkbox beside each vulnerability you want to update the status of. To select all, select the checkbox in the table header. 1. In the **Set status** dropdown list, select the desired status. +1. If the **Dismissed** status is chosen, select the desired reason in the **Set dismissal reason** dropdown list. +1. In the **Add a comment** input, you can provide a comment. For the **Dismissed** status, a comment is required. 1. Select **Change status**. -![Project Vulnerability Report](img/project_security_dashboard_status_change_v14_2.png) +![Project Vulnerability Report](img/project_security_dashboard_status_change_v16_0.png) ## Sort vulnerabilities by date detected @@ -242,7 +248,7 @@ Vulnerability records cannot be deleted, so a permanent record always remains. You can dismiss a vulnerability in projects and groups: 1. Select the vulnerability in the Security Dashboard. -1. In the upper right, from the **Status** selector menu, select **Dismissed**. +1. In the upper-right corner, from the **Status** dropdown list, select **Dismissed**. 1. Optional. Add a reason for the dismissal and select **Save comment**. To undo this action, select a different status from the same menu. @@ -256,7 +262,7 @@ To undo this action, select a different status from the same menu. To add a new vulnerability finding from your project level Vulnerability Report page: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Vulnerability Report**. +1. On the left sidebar, select **Security and Compliance > Vulnerability Report**. 1. Select **Submit vulnerability**. 1. Complete the fields and submit the form. diff --git a/doc/user/award_emojis.md b/doc/user/award_emojis.md index e1459717241..c3aad86461b 100644 --- a/doc/user/award_emojis.md +++ b/doc/user/award_emojis.md @@ -4,18 +4,28 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Award emojis **(FREE)** +# Emoji reactions **(FREE)** + +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emojis" to "emoji reactions" in GitLab 16.0. +> - Reacting with emojis on work items (such as tasks, objectives, and key results) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393599) in GitLab 16.0. When you're collaborating online, you get fewer opportunities for high-fives -and thumbs-ups. Emojis can be awarded to [issues](project/issues/index.md), [merge requests](project/merge_requests/index.md), -[snippets](snippets.md), and anywhere you can have a thread. +and thumbs-ups. React with emojis on: + +- [Issues](project/issues/index.md). +- [Tasks](tasks.md). +- [Merge requests](project/merge_requests/index.md), +[snippets](snippets.md). +- [Epics](../user/group/epics/index.md). +- [Objectives and key results](okrs.md). +- Anywhere else you can have a comment thread. -![Award emoji](img/award_emoji_select_v14_6.png) +![Emoji reactions](img/award_emoji_select_v14_6.png) -Award emojis make it much easier to give and receive feedback without a long +Emoji reactions make it much easier to give and receive feedback without a long comment thread. -For information on the relevant API, see [Award Emoji API](../api/award_emoji.md). +For information on the relevant API, see [Emoji reactions API](../api/award_emoji.md). ## Sort issues and merge requests on vote count @@ -29,17 +39,17 @@ The total number of votes is not summed up. An issue with 18 upvotes and 5 downvotes is considered more popular than an issue with 17 upvotes and no downvotes. -## Award emojis for comments +## Emoji reactions for comments -Award emojis can also be applied to individual comments when you want to +Emoji reactions can also be applied to individual comments when you want to celebrate an accomplishment or agree with an opinion. -To add an award emoji: +To add an emoji reaction: -1. In the upper right of the comment, select the smile (**{slight-smile}**). -1. Select an emoji from the dropdown list. +1. In the upper-right corner of the comment, select the smile (**{slight-smile}**). +1. Select an emoji from the emoji picker. -To remove an award emoji, select the emoji again. +To remove an emoji reaction, select the emoji again. ## Custom emojis diff --git a/doc/user/clusters/agent/ci_cd_workflow.md b/doc/user/clusters/agent/ci_cd_workflow.md index 982411d35f9..7a3f09f50ca 100644 --- a/doc/user/clusters/agent/ci_cd_workflow.md +++ b/doc/user/clusters/agent/ci_cd_workflow.md @@ -1,13 +1,13 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Using GitLab CI/CD with a Kubernetes cluster **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. -> - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. +> - The pre-configured variable `$KUBECONFIG` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. > - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. @@ -78,7 +78,8 @@ To authorize the agent to access the GitLab project where you keep Kubernetes ma - You can install additional agents into the same cluster to accommodate additional hierarchies. - You can authorize up to 100 projects. -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +All CI/CD jobs now include a `kubeconfig` file with contexts for every shared agent connection. +The `kubeconfig` path is available in the environment variable `$KUBECONFIG`. Choose the context to run `kubectl` commands from your CI/CD scripts. ### Authorize the agent to access projects in your groups @@ -104,7 +105,8 @@ To authorize the agent to access all of the GitLab projects in a group or subgro - You can authorize up to 100 groups. All the projects that belong to the group and its subgroups are now authorized to access the agent. -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +All CI/CD jobs now include a `kubeconfig` file with contexts for every shared agent connection. +The `kubeconfig` path is available in an environment variable `$KUBECONFIG`. Choose the context to run `kubectl` commands from your CI/CD scripts. ## Update your `.gitlab-ci.yml` file to run `kubectl` commands @@ -159,10 +161,10 @@ When you deploy to an environment that has both a - The certificate-based cluster's context is called `gitlab-deploy`. This context is always selected by default. -- In GitLab 14.9 and later, agent contexts are included in the - `KUBECONFIG`. You can select them by using `kubectl config use-context path/to/agent/repository:agent-name`. +- In GitLab 14.9 and later, agent contexts are included in `$KUBECONFIG`. + You can select them by using `kubectl config use-context path/to/agent/repository:agent-name`. - In GitLab 14.8 and earlier, you can still use agent connections, but for environments that - already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. + already have a certificate-based cluster, the agent connections are not included in `$KUBECONFIG`. To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` configuration context. For example: @@ -183,6 +185,17 @@ deploy: # ... rest of your job configuration ``` +### Environments with KAS that use self-signed certificates + +If you use an environment with KAS and a self-signed certificate, you must configure your Kubernetes client to trust the certificate authority (CA) that signed your certificate. + +To configure your client, do one of the following: + +- Set a CI/CD variable `SSL_CERT_FILE` with the KAS certificate in PEM format. +- Configure the Kubernetes client with `--certificate-authority=$KAS_CERTIFICATE`, where `KAS_CERTIFICATE` is a CI/CD variable with the CA certificate of KAS. +- Place the certificates in an appropriate location in the job container by updating the container image or mounting via the runner. +- Not recommended. Configure the Kubernetes client with `--insecure-skip-tls-verify=true`. + ## Restrict project and group access by using impersonation **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. @@ -340,3 +353,28 @@ If you attempt to use `kubectl` without TLS, you might get an error like: $ kubectl get pods error: You must be logged in to the server (the server has asked for the client to provide credentials) ``` + +### Unable to connect to the server: certificate signed by unknown authority + +If you use an environment with KAS and a self-signed certificate, your `kubectl` call might return this error: + +```plaintext +kubectl get pods +Unable to connect to the server: x509: certificate signed by unknown authority +``` + +The error occurs because the job does not trust the certificate authority (CA) that signed the KAS certificate. + +To resolve the issue, [configure `kubectl` to trust the CA](#environments-with-kas-that-use-self-signed-certificates). + +### Validation errors + +If you use `kubectl` versions v1.27.0 or v.1.27.1, you might get the following error: + +```plaintext +error: error validating "file.yml": error validating data: the server responded with the status code 426 but did not return more information; if you choose to ignore these errors, turn validation off with --validate=false +``` + +This issue is caused by [a bug](https://github.com/kubernetes/kubernetes/issues/117463) with `kubectl` and other tools that use the shared Kubernetes libraries. + +To resolve the issue, use another version of `kubectl`. diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md index 787b0062017..106f751555a 100644 --- a/doc/user/clusters/agent/gitops.md +++ b/doc/user/clusters/agent/gitops.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -12,6 +12,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/346585) to make the `id` attribute optional in GitLab 15.7. > - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. +NOTE: +From GitLab 15.10, you should use [Flux](gitops/flux.md) for GitOps. For more information, see +[this announcement blog post](https://about.gitlab.com/blog/2023/02/08/why-did-we-choose-to-integrate-fluxcd-with-gitlab/). + With GitOps, you can manage containerized clusters and applications from a Git repository that: - Is the single source of truth of your system. diff --git a/doc/user/clusters/agent/gitops/example_repository_structure.md b/doc/user/clusters/agent/gitops/example_repository_structure.md new file mode 100644 index 00000000000..93c05e40bfd --- /dev/null +++ b/doc/user/clusters/agent/gitops/example_repository_structure.md @@ -0,0 +1,78 @@ +--- +stage: Deploy +group: Environments +info: An example of how to structure a repository for GitOps deployments +--- + +# Example GitOps repository structure **(FREE)** + +This page describes an example structure for a project that builds and deploys an application +to a Kubernetes cluster with [GitOps](https://about.gitlab.com/topics/gitops) and the +[GitLab agent for Kubernetes](../../agent/gitops.md). + +You can find an example project that uses this structure +[in this GitLab repository](https://gitlab.com/tigerwnz/minimal-gitops-app). You can use the example project +as a starting point to create your own deployment project. + +## Deployment workflow + +The default branch is the single source of truth for your application and the +Kubernetes manifests that deploy it. To be reflected in a Kubernetes cluster, +a code or configuration change must exist in the default branch. + +A GitLab agent for Kubernetes is installed in every Kubernetes cluster. The agent +is configured to sync manifests from a corresponding branch in the repository. +These branches represent the state of each cluster, and contain only commits that +exist in the default branch. + +Changes are deployed by merging the default branch into the branch of a cluster. +The agent that watches the branch picks up the change and syncs it to the cluster. + +For the actual deployment, the example project uses the GitLab agent for Kubernetes, +but you can also use other GitOps tools. + +### Review apps + +Ephemeral environments such as [review apps](../../../../ci/review_apps/index.md) +are deployed differently. Their configuration does not exist on the default branch, +and the changes are not meant to be deployed to a permanent environment. Review app +manifests are generated and deployed in a merge request feature branch, which is removed +when the MR is merged. + +## Example deployment + +The example project deploys to two permanent environments, staging and production, +which each have a dedicated Kubernetes cluster. A third cluster is used for ephemeral +review apps. + +Each cluster has a corresponding branch that represents the current state of the cluster: +`_gitlab/agents/staging`, `_gitlab/agents/production` and `_gitlab/agents/review`. Each branch is +[protected](../../../../user/project/protected_branches.md) and +a [project access token](../../../../user/project/settings/project_access_tokens.md) +is created for each branch with a configuration that allows only the corresponding token to push to the branch. +This ensures that environment branches are updated only through the configured process. + +Deployment branches are updated by CI/CD jobs. The access token that allows pushing to each +branch is configured as a [CI/CD variable](../../../../ci/variables/index.md). These variables +are protected, and only available to pipelines running on a protected branch. +The CI/CD job merges the default branch `main` into the deployment branch, and pushes +the deployment branch back to the repository using the provided token. To preserve the +commit history between both branches, the CI/CD job uses a fast-forward merge. + +Each cluster has an agent for Kubernetes, and each agent is configured to +[sync manifests from the branch corresponding to its cluster](../gitops.md#gitops-configuration-reference). +In your own project, you can different GitOps tool like Flux, or use the same configuration to deploy +to virtual machines with GitLab CI/CD. + +### Application changes + +The example project follows this process to deploy an application change: + +1. A new feature branch is created with the desired changes. The pipeline builds an image, + runs the test suite, and deploy the changes to a review app in the `review` cluster. +1. The feature branch is merged to `main` and the review app is removed. +1. Manifests are updated on `main` (either directly or via merge request) to point to an updated + version of the deployed image. The pipeline automatically merges `main` into the `_gitlab/agents/staging` + branch, which updates the `staging` cluster. +1. The `production` job is triggered manually, and merges `main` into the `_gitlab/agents/production` branch, + deploying to the `production` cluster. diff --git a/doc/user/clusters/agent/gitops/flux.md b/doc/user/clusters/agent/gitops/flux.md new file mode 100644 index 00000000000..98840080716 --- /dev/null +++ b/doc/user/clusters/agent/gitops/flux.md @@ -0,0 +1,36 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Flux (Beta) **(FREE)** + +Flux is a GitOps tool that helps you manage your Kubernetes clusters. +You can use Flux to: + +- Keep your clusters in sync with your Git repositories. +- Reconcile code changes with your deployments. +- Manage your Flux installation itself with a bootstrap. + +To get started, see the [Flux installation documentation](https://fluxcd.io/flux/installation). + +Support for Flux is in [Beta](../../../../policy/alpha-beta-support.md#beta). + +## Bootstrap installation + +Use the Flux command [`bootstrap gitlab`](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise) +to configure a Kubernetes cluster to manage itself from a Git repository. + +You must authenticate your installation with either: + +- Recommended. [A project access token](../../../project/settings/project_access_tokens.md). +- A [group access token](../../../group/settings/group_access_tokens.md). +- A [personal access token](../../../profile/personal_access_tokens.md). + +Some Flux features like [automated image updates](https://fluxcd.io/flux/guides/image-update/) require +write access to the source repositories. + +## GitOps repository structure + +You should organize your repositories to meet the needs of your team. For detailed recommendations, see the Flux [repository structure documentation](https://fluxcd.io/flux/guides/repository-structure/). diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md new file mode 100644 index 00000000000..bdb772ac14e --- /dev/null +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -0,0 +1,192 @@ +--- +stage: Deploy +group: Environments +info: A tutorial using Flux +--- + +# Tutorial: Set up Flux for GitOps **(FREE)** + +This tutorial teaches you how to set up Flux for GitOps. You'll set up a sample project, +complete a bootstrap Flux installation, and authenticate your installation with a +[project deploy token](../../../project/deploy_tokens/index.md). + +You can find the fully configured tutorial project [in this GitLab repository](https://gitlab.com/gitlab-org/configure/examples/flux/flux-config). +It works in conjunction with [this repository](https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests/-/tree/main), which contains the example Kubernetes manifests. + +To set up Flux for GitOps: + +1. [Create a personal access token](#create-a-personal-access-token) +1. [Create the Flux repository](#create-the-flux-repository) +1. [Create the Kubernetes manifest repository](#create-the-kubernetes-manifest-repository) +1. [Configure Flux to sync your manifests](#configure-flux-to-sync-your-manifests) +1. [Verify your configuration](#verify-your-configuration) + +Prerequisites: + +- You must have a Kubernetes cluster running. + +## Create a personal access token + +To authenticate with the Flux CLI, you must create a personal access token +with the `api` scope: + +1. In the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Access Tokens**. +1. Enter a name and optional expiry date for the token. +1. Select the `api` scope. +1. Select **Create personal access token**. + +You can also use a [project](../../../project/settings/project_access_tokens.md) or [group access token](../../../group/settings/group_access_tokens.md) with the `api` scope. + +## Create the Flux repository + +Create a Git repository, install Flux, and authenticate Flux with your repo: + +1. Make sure your `kubectl` is configured to access your cluster. +1. [Install the Flux CLI](https://fluxcd.io/flux/installation/#install-the-flux-cli). You must install Flux v2 or higher. +1. In GitLab, create a new empty project called `flux-config`. +1. From your shell, export a `GITLAB_TOKEN` environment variable with the value of your personal access token. + For example, `export GITLAB_TOKEN=<personal-access-token>`. +1. Run the `bootstrap` command. The exact command depends on whether you are + creating the Flux repository under a GitLab user, group, or subgroup. For more information, + see the [Flux bootstrap documentation](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise). + + In this tutorial, you're working with a public project in a subgroup. The bootstrap command looks like this: + + ```shell + flux bootstrap gitlab \ + --owner=gitlab-org/configure/examples/flux \ + --repository=flux-config \ + --branch=main \ + --path=clusters/my-cluster \ + --deploy-token-auth + ``` + + This command installs Flux on the Kubernetes cluster and configures it to manage itself from the repository `flux-config`. + The command also automatically creates the project deploy token required to access the `flux-config` repository. + +Great work! You now have a repository bootstrapped with a Flux configuration. Any updates to your repository are automatically synced to the cluster. + +## Create the Kubernetes manifest repository + +Next, create a repository for your Kubernetes manifests: + +1. In GitLab, create a new repository called `web-app-manifests`. +1. Add a file to `web-app-manifests` named `nginx-deployment.yaml` with the following contents: + + ```yaml + apiVersion: apps/v1 + + kind: Deployment + + metadata: + name: nginx-deployment + labels: + app: nginx + spec: + replicas: 3 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + ``` + +1. In the new repository, [create a deploy token](../../../project/deploy_tokens/index.md#create-a-deploy-token) with only the `read_repository` scope. +1. Store your deploy token username and password somewhere safe. +1. In Flux CLI, create a secret with your deploy token and point the secret to the new repository. For example: + + ```shell + flux create secret git flux-deploy-authentication \ + --url=https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests \ + --namespace=default \ + --username=<token-user-name> \ + --password=<token-password> + ``` + +1. To check if your secret was generated successfully, run: + + ```shell + kubectl -n default get secrets flux-deploy-authentication -o yaml + ``` + + Under `data`, you should see base64-encoded values associated with your token username and password. + +Congratulations! You now have a manifest repository, a deploy token, and a secret generated directly on your cluster. + +## Configure Flux to sync your manifests + +Next, tell `flux-config` to sync with the `web-app-manifests` repository. + +To do so, create a [`GitRepository`](https://fluxcd.io/flux/components/source/gitrepositories/) resource: + +1. Clone the `flux-config` repo to your machine. +1. In your local clone of `flux-config`, add the `GitRepository` file `clusters/my-cluster/web-app-manifests-source.yaml`: + + ```yaml + --- + apiVersion: source.toolkit.fluxcd.io/v1beta2 + kind: GitRepository + metadata: + name: web-app-manifests + namespace: default + spec: + interval: 1m0s + ref: + branch: main + secretRef: + name: flux-deploy-authentication + url: https://gitlab.com/gitlab-org/configure/examples/flux/web-app-manifests + ``` + + This file uses `secretRef` to refer back to the deploy token secret you created in the last step. + +1. In your local clone of `flux-config`, add the `GitRepository` file `clusters/my-cluster/web-app-manifests-kustomization.yaml`: + + ```yaml + --- + apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 + kind: Kustomization + metadata: + name: nginx-source-kustomization + namespace: default + spec: + interval: 1m0s + path: ./ + prune: true + sourceRef: + kind: GitRepository + name: web-app-manifests + namespace: default + targetNamespace: default + ``` + + This file adds a [`Kustomization`](https://fluxcd.io/flux/components/kustomize/kustomization/) resource that tells Flux to sync the manifests from + `web-app-manifests` with `kustomize`. + +1. Commit the new files and push. + +## Verify your configuration + +You should see a newly created `nginx-deployment` pod in your cluster. + +To check whether the `nginx-deployment` pod is running in the default namespace, run the following: + +```shell +kubectl -n default get pods -n default +``` + +If you want to see the deployment sync again, try updating the number of replicas in the +`nginx-deployment.yaml` file and push to your `main` branch. If all is working well, it +should sync to the cluster. + +Excellent work! You've successfully set up a complete Flux project. diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md index b9a59d37f5d..182fd87e6f9 100644 --- a/doc/user/clusters/agent/gitops/helm.md +++ b/doc/user/clusters/agent/gitops/helm.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Using Helm charts to update a Kubernetes cluster (Alpha) **(FREE)** +# Using Helm charts to update a Kubernetes cluster (Experiment) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371019) in GitLab 15.4. > - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. @@ -13,11 +13,11 @@ You can deploy Helm charts to your Kubernetes cluster and keep the resources in with your charts and values. To do this, you use the pull-based GitOps features of the agent for Kubernetes. -This feature is in Alpha and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) +This feature is an Experiment and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) to track future work. Tell us about your use cases by leaving comments in the epic. NOTE: -This feature is Alpha. In future releases, to accommodate new features, the configuration format might change without notice. +This feature is an Experiment. In future releases, to accommodate new features, the configuration format might change without notice. ## GitOps workflow steps diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md index dc1cbe3009d..6e1b7da9c6c 100644 --- a/doc/user/clusters/agent/gitops/secrets_management.md +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -50,9 +50,9 @@ To deploy containers from the GitLab Container Registry, you must configure the 1. Generate a GitLab token with at least `read-registry` rights. The token can be either a Personal or a Project Access Token. 1. Create a Kubernetes secret manifest YAML file. Update the values as needed: - ```shell - kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml - ``` + ```shell + kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml + ``` 1. Encrypt the secret into a `SealedSecret` manifest: diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 8f4855a7f93..ccb3a346903 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -12,6 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the GitLab agent became available on GitLab.com. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Renamed](https://gitlab.com/groups/gitlab-org/-/epics/7167) from "GitLab Kubernetes Agent" to "GitLab agent for Kubernetes" in GitLab 14.6. +> - Flux [recommended](https://gitlab.com/gitlab-org/gitlab/-/issues/357947#note_1253489000) as GitOps solution in GitLab 15.10. You can connect your Kubernetes cluster with GitLab to deploy, manage, and monitor your cloud-native solutions. @@ -33,20 +34,7 @@ You can choose from two primary workflows. The GitOps workflow is recommended. ### GitOps workflow -In a [**GitOps** workflow](gitops.md): - -- You keep your Kubernetes manifests in GitLab. -- You install a GitLab agent in your cluster. -- Any time you update your manifests, the agent updates the cluster. -- The cluster automatically cleans up unexpected changes. It uses - [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/) - to fix any configuration inconsistencies that third parties introduce. - -This workflow is fully driven with Git and is considered **pull-based**, -because the cluster is pulling updates from your GitLab repository. - -GitLab recommends this workflow. We are actively investing in this workflow -so we can provide a first-class experience. +You should use Flux for GitOps. To get started, see the GitLab [Flux documentation](../../../user/clusters/agent/gitops/flux.md). ### GitLab CI/CD workflow @@ -64,14 +52,14 @@ Use this workflow: This workflow has a weaker security model and is not recommended for production deployments. -## Supported cluster versions +## GitLab agent for Kubernetes supported cluster versions -GitLab supports the following Kubernetes versions. You can upgrade your +The agent for Kubernetes supports the following Kubernetes versions. You can upgrade your Kubernetes version to a supported version at any time: +- 1.26 (support ends on March 22, 2024 or when 1.29 becomes supported) - 1.25 (support ends on October 22, 2023 or when 1.28 becomes supported) - 1.24 (support ends on July 22, 2023 or when 1.27 becomes supported) -- 1.23 (support ends on February 22, 2023 or when 1.26 becomes supported) GitLab aims to support a new minor Kubernetes version three months after its initial release. GitLab supports at least three production-ready Kubernetes minor versions at any given time. diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 297210ab8ef..1bcbb42fc8e 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -138,6 +138,23 @@ By default, the Helm installation command generated by GitLab: To see the full list of customizations available, see the Helm chart's [default values file](https://gitlab.com/gitlab-org/charts/gitlab-agent/-/blob/main/values.yaml). +##### Use the agent when KAS is behind a self-signed certificate + +When [KAS](../../../../administration/clusters/kas.md) is behind a self-signed certificate, +you can set the value of `config.caCert` to the certificate. For example: + +```shell +helm update --install gitlab-agent gitlab/gitlab-agent \ + --set-file config.caCert=my-custom-ca.pem +``` + +In this example, `my-custom-ca.pem` is the path to a local file that contains +the CA certificate used by KAS. The certificate is automatically stored in a +config map and mounted in the `agentk` pod. + +If KAS is installed with the GitLab chart, and the chart is configured to provide +an [auto-generated self-signed wildcard certificate](https://docs.gitlab.com/charts/installation/tls.html#option-4-use-auto-generated-self-signed-wildcard-certificate), you can extract the CA certificate from the `RELEASE-wildcard-tls-ca` secret. + ##### Use the agent behind an HTTP proxy > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351867) in GitLab 15.0, the GitLab agent Helm chart supports setting environment variables. @@ -155,6 +172,10 @@ helm upgrade --install gitlab-agent gitlab/gitlab-agent \ ... ``` +NOTE: +DNS rebind protection is disabled when either the `HTTP_PROXY` or the `HTTPS_PROXY` environment variable is set, +and the domain DNS can't be resolved. + #### Advanced installation method GitLab also provides a [KPT package for the agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/build/deployment/gitlab-agent). This method provides greater flexibility, but is only recommended for advanced users. diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index f5f235d2758..531dac20fb6 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -107,71 +107,72 @@ This error occurs when your GitLab instance is using a certificate signed by an certificate authority that is unknown to the agent. To fix this issue, you can present the CA certificate file to the agent -by using a Kubernetes `configmap` and mount the file in the agent `/etc/ssl/certs` directory from where it -is picked up automatically. +by [customizing the Helm installation](install/index.md#customize-the-helm-installation). +Add `--set config.caCert="$(cat ~/path/to/ca.crt)"` to the `helm install` command. Make sure to replace `~/path/to/ca.crt` +with the path to your internal CA's certificate file. The file should be a valid PEM or DER-encoded certificate. -For example, if your internal CA certificate is `myCA.pem`: +When you deploy `agentk` with a set `config.caCert` value, the certificate is added to `configmap` and the certificate file is mounted in `/etc/ssl/certs`. -```plaintext -kubectl -n gitlab-agent create configmap ca-pemstore --from-file=myCA.pem +```yaml +$ kubectl get configmap -lapp=gitlab-agent -o yaml +apiVersion: v1 +items: +- apiVersion: v1 + data: + ca.crt: |- + -----BEGIN CERTIFICATE----- + MIIFmzCCA4OgAwIBAgIUE+FvXfDpJ869UgJitjRX7HHT84cwDQYJKoZIhvcNAQEL + ...truncated certificate... + GHZCTQkbQyUwBWJOUyOxW1lro4hWqtP4xLj8Dpq1jfopH72h0qTGkX0XhFGiSaM= + -----END CERTIFICATE----- + kind: ConfigMap + metadata: + annotations: + meta.helm.sh/release-name: self-signed + meta.helm.sh/release-namespace: gitlab-agent-self-signed + creationTimestamp: "2023-03-07T20:12:26Z" + labels: + app: gitlab-agent + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: gitlab-agent + app.kubernetes.io/version: v15.9.0 + helm.sh/chart: gitlab-agent-1.11.0 + name: self-signed-gitlab-agent + resourceVersion: "263184207" +kind: List ``` -Then in `resources.yml`: +You might see a similar error in the [agent server (KAS) logs](../../../administration/logs/index.md#gitlab-agent-server) of your GitLab application server: -```yaml - spec: - serviceAccountName: gitlab-agent - containers: - - name: agent - image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:<version>" - args: - - --token-file=/config/token - - --kas-address - - wss://kas.host.tld:443 # replace this line with the line below if using Omnibus GitLab or GitLab.com. - # - wss://gitlab.host.tld:443/-/kubernetes-agent/ - # - wss://kas.gitlab.com # for GitLab.com users, use this KAS. - # - grpc://host.docker.internal:8150 # use this attribute when connecting from Docker. - volumeMounts: - - name: token-volume - mountPath: /config - - name: ca-pemstore-volume - mountPath: /etc/ssl/certs/myCA.pem - subPath: myCA.pem - volumes: - - name: token-volume - secret: - secretName: gitlab-agent-token - - name: ca-pemstore-volume - configMap: - name: ca-pemstore - items: - - key: myCA.pem - path: myCA.pem +```json +{"level":"error","time":"2023-03-07T20:19:48.151Z","msg":"AgentInfo()","grpc_service":"gitlab.agent.agent_configuration.rpc.AgentConfiguration","grpc_method":"GetConfiguration","error":"Get \"https://gitlab.example.com/api/v4/internal/kubernetes/agent_info\": x509: certificate signed by unknown authority"} ``` -Alternatively, you can mount the certificate file at a different location and specify it for the -`--ca-cert-file` agent parameter: +To fix it, [install your internal CA's public certificate](https://docs.gitlab.com/omnibus/settings/ssl/#install-custom-public-certificates) in the `/etc/gitlab/trusted-certs` directory. -```yaml - containers: - - name: agent - image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:<version>" - args: - - --ca-cert-file=/tmp/myCA.pem - - --token-file=/config/token - - --kas-address - - wss://kas.host.tld:443 # replace this line with the line below if using Omnibus GitLab or GitLab.com. - # - wss://gitlab.host.tld:443/-/kubernetes-agent/ - # - wss://kas.gitlab.com # for GitLab.com users, use this KAS. - # - grpc://host.docker.internal:8150 # use this attribute when connecting from Docker. - volumeMounts: - - name: token-volume - mountPath: /config - - name: ca-pemstore-volume - mountPath: /tmp/myCA.pem - subPath: myCA.pem +Alternatively, you can configure the agent server (KAS) to read the certificate from a custom directory. +Add the following configuration to `/etc/gitlab/gitlab.rb`: + +```ruby +gitlab_kas['env'] = { + 'SSL_CERT_DIR' => "/opt/gitlab/embedded/ssl/certs/" + } ``` +To apply the changes: + +1. Reconfigure GitLab. + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Restart `gitlab-kas`. + + ```shell + gitlab-ctl restart gitlab-kas + ``` + ## Project not found ```json @@ -238,4 +239,4 @@ When you install the agent, you might encounter an error that states: Error: parse error at (gitlab-agent/templates/observability-secret.yaml:1): unclosed action ``` -This error is typically caused by an incompatible version of Helm. To resolve the issue, ensure that you are using a version of Helm [compatible with your version of Kubernetes](index.md#supported-cluster-versions). +This error is typically caused by an incompatible version of Helm. To resolve the issue, ensure that you are using a version of Helm [compatible with your version of Kubernetes](index.md#gitlab-agent-for-kubernetes-supported-cluster-versions). diff --git a/doc/user/clusters/agent/work_with_agent.md b/doc/user/clusters/agent/work_with_agent.md index 0b0e1365604..2d54f67724e 100644 --- a/doc/user/clusters/agent/work_with_agent.md +++ b/doc/user/clusters/agent/work_with_agent.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/cost_management.md b/doc/user/clusters/cost_management.md index 74bfab49c55..6ed0f08c4a7 100644 --- a/doc/user/clusters/cost_management.md +++ b/doc/user/clusters/cost_management.md @@ -1,81 +1,12 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../index.md' --- -# Cluster cost management (DEPRECATED) **(ULTIMATE)** +# Cluster cost management (removed) **(ULTIMATE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216737) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. - -Cluster cost management provides insights into cluster resource usage. GitLab provides an example -[`kubecost-cost-model`](https://gitlab.com/gitlab-examples/kubecost-cost-model/) -project that uses the GitLab Prometheus integration and -[OpenCost `cost-model`](https://github.com/opencost/opencost) to provide cluster cost -insights within GitLab: - -![Example dashboard](img/kubecost_v13_5.png) - -## Configure cluster cost management - -To get started with cluster cost management, you need the Maintainer role for a project or group. - -1. Clone the [`kubecost-cost-model`](https://gitlab.com/gitlab-examples/kubecost-cost-model/) - example repository, which contains minor modifications to the upstream Kubecost - `cost-model` project: - - Configures your Prometheus endpoint to the GitLab-managed Prometheus. You may - need to change this value if you use a non-managed Prometheus. - - Adds the necessary annotations to the deployment manifest to be scraped by - GitLab-managed Prometheus. - - Changes the Google Pricing API access key to the GitLab access key. - - Contains definitions for a custom GitLab Metrics dashboard to show the cost insights. -1. Connect GitLab with Prometheus, depending on your configuration: - - *If Prometheus is already configured,* navigate to **Settings > Integrations > Prometheus** - to provide the API endpoint of your Prometheus server. - - *To use the Prometheus cluster integration,* navigate to your cluster's **Details** page, - select the **Integrations** tab, and follow the instructions to enable the Prometheus - cluster integration. -1. Set up the Prometheus integration on the cloned example project. -1. Add the Kubecost `cost-model` to your cluster: - - *For non-managed clusters*, deploy it with GitLab CI/CD. - - *To deploy it manually*, use the following commands: - - ```shell - kubectl create namespace cost-model - kubectl apply -f kubernetes/ --namespace cost-model - ``` - -To access the cost insights, navigate to **Monitor > Metrics** and select -the `default_costs.yml` dashboard. You can [customize](#customize-the-cost-dashboard) -this dashboard. - -### Customize the cost dashboard - -You can customize the cost dashboard by editing the `.gitlab/dashboards/default_costs.yml` -file or creating similar dashboard configuration files. For more information, see -[Custom dashboards](../../operations/metrics/dashboards/index.md). - -#### Available metrics - -Metrics contain both instance and node labels. The instance label is scheduled for deprecation in a future version. - -- `node_cpu_hourly_cost` - Hourly cost per vCPU on this node. -- `node_gpu_hourly_cost` - Hourly cost per GPU on this node. -- `node_ram_hourly_cost` - Hourly cost per gigabyte of memory on this node. -- `node_total_hourly_cost` - Total node cost per hour. -- `container_cpu_allocation` - Average number of CPUs requested/used over the previous minute. -- `container_gpu_allocation` - Average number of GPUs requested over the previous minute. -- `container_memory_allocation_bytes` - Average bytes of RAM requested/used over the previous minute. -- `pod_pvc_allocation` - Bytes provisioned for a PVC attached to a pod. -- `pv_hourly_cost` - Hourly cost per GB on a persistent volume. - -Some examples are provided in the -[`kubecost-cost-model` repository](https://gitlab.com/gitlab-examples/kubecost-cost-model/-/blob/master/PROMETHEUS.md#example-queries). +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/clusters/create/index.md b/doc/user/clusters/create/index.md index 0721cea965a..57526394301 100644 --- a/doc/user/clusters/create/index.md +++ b/doc/user/clusters/create/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index f4313656803..c6a61f58974 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Cluster Environments (DEPRECATED) **(PREMIUM)** +# Cluster Environments (deprecated) **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13392) in GitLab 12.3 for group-level clusters. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14809) in GitLab 12.4 for instance-level clusters. diff --git a/doc/user/clusters/integrations.md b/doc/user/clusters/integrations.md index c5e56fcd3a7..3076730b10f 100644 --- a/doc/user/clusters/integrations.md +++ b/doc/user/clusters/integrations.md @@ -1,100 +1,12 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../index.md' --- -# Cluster integrations (DEPRECATED) **(FREE)** +# Cluster integrations (removed) **(FREE)** -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. - -GitLab provides several ways to integrate applications to your -Kubernetes cluster. - -To enable cluster integrations, first add a Kubernetes cluster to a GitLab -[project](../project/clusters/index.md) or -[group](../group/clusters/index.md) or -[instance](../instance/clusters/index.md). - -You can install your applications manually as shown in the following sections, or use the -[Cluster management project template](management_project_template.md) that automates the -installation. - -Although, the [Cluster management project template](management_project_template.md) still -requires that you manually do the last steps of this section, -[Enable Prometheus integration for your cluster](#enable-prometheus-integration-for-your-cluster). [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/326565) -to automate this step. - -Prometheus cluster integrations can only be enabled for clusters [connected through cluster certificates](../project/clusters/add_existing_cluster.md). - -To enable Prometheus for your cluster connected through the [GitLab agent](agent/index.md), you can [integrate it manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). - -There is no option to enable Elastic Stack for your cluster if it is connected with the GitLab agent. -Follow this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/300230) for updates. - -## Prometheus cluster integration - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55244) in GitLab 13.11. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. However, you can **still use** Prometheus -for Kubernetes clusters connected to GitLab through the -[agent](agent/index.md) by [enabling Prometheus manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). - -You can integrate your Kubernetes cluster with -[Prometheus](https://prometheus.io/) for monitoring key metrics of your -apps directly from the GitLab UI. - -Once enabled, you can see metrics from services available in the -[metrics library](../project/integrations/prometheus_library/index.md). - -### Prometheus Prerequisites - -To use this integration: - -1. Prometheus must be installed in your cluster in the `gitlab-managed-apps` namespace. -1. The `Service` resource for Prometheus must be named `prometheus-prometheus-server`. - -You can manage your Prometheus however you like, but as an example, you can set -it up using [Helm](https://helm.sh/) as follows: - -```shell -# Create the required Kubernetes namespace -kubectl create ns gitlab-managed-apps - -# Download Helm chart values that is compatible with the requirements above. -# These are included in the Cluster Management project template. -wget https://gitlab.com/gitlab-org/project-templates/cluster-management/-/raw/master/applications/prometheus/values.yaml - -# Add the Prometheus community Helm chart repository -helm repo add prometheus-community https://prometheus-community.github.io/helm-charts - -# Install Prometheus -helm install prometheus prometheus-community/prometheus -n gitlab-managed-apps --values values.yaml -``` - -Alternatively, you can use your preferred installation method to install -Prometheus as long as you meet the requirements above. - -### Enable Prometheus integration for your cluster - -To enable the Prometheus integration for your cluster: - -1. Go to the cluster's page: - - For a [project-level cluster](../project/clusters/index.md), go to your project's - **Infrastructure > Kubernetes clusters**. - - For a [group-level cluster](../group/clusters/index.md), go to your group's - **Kubernetes** page. - - For an [instance-level cluster](../instance/clusters/index.md), go to your instance's - **Kubernetes** page. -1. Select the **Integrations** tab. -1. Check the **Enable Prometheus integration** checkbox. -1. Select **Save changes**. -1. Go to the **Health** tab to see your cluster's metrics. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 18317ae09ed..0ad3fdbef2e 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Cluster management project (DEPRECATED) **(FREE)** +# Cluster management project (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32810) in GitLab 12.5. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index 9edb012ba92..4b6c8dcac4c 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index e5d81091094..fde8e455421 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Migrate from GitLab Managed Apps to Cluster Management Projects (DEPRECATED) **(FREE)** +# Migrate from GitLab Managed Apps to Cluster Management Projects (deprecated) **(FREE)** The GitLab Managed Apps were deprecated in GitLab 14.0 in favor of user-controlled Cluster Management projects. diff --git a/doc/user/compliance/compliance_report/index.md b/doc/user/compliance/compliance_report/index.md index 04609026793..d04aeec066f 100644 --- a/doc/user/compliance/compliance_report/index.md +++ b/doc/user/compliance/compliance_report/index.md @@ -5,38 +5,58 @@ group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Compliance report **(ULTIMATE)** +# Compliance reports **(ULTIMATE)** + +See reports about compliance violations and compliance frameworks for the group. + +## Compliance violations report > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36524) in GitLab 12.8 as Compliance Dashboard. +> - Compliance violation drawer [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299357) in GitLab 14.1. > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/299360) to compliance report in GitLab 14.2. > - [Replaced](https://gitlab.com/groups/gitlab-org/-/epics/5237) by merge request violations in GitLab 14.6 [with a flag](../../../administration/feature_flags.md) named `compliance_violations_report`. Disabled by default. > - GraphQL API [introduced](https://gitlab.com/groups/gitlab-org/-/epics/7222) in GitLab 14.9. > - [Generally available](https://gitlab.com/groups/gitlab-org/-/epics/5237) in GitLab 14.10. [Feature flag `compliance_violations_report`](https://gitlab.com/gitlab-org/gitlab/-/issues/346266) removed. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112111) to compliance violations report in GitLab 15.9. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/394950) ability to create/edit compliance frameworks in GitLab 16.0. -Compliance report gives you the ability to see a group's merge request activity. It provides a -high-level view for all projects in the group. For example, code approved for merging into -production. +With compliance violations report, you can see a high-level view of merge request activity for all projects in the group. -You can use the report to get: +When you select a row in the compliance report, a drawer appears that provides: -- A list of compliance violations from all merged merge requests within the group. -- The reason and severity of each compliance violation. -- A link to the merge request that caused each compliance violation. +- The project name and [compliance framework label](../../project/settings/index.md#add-a-compliance-framework-to-a-project), + if the project has one assigned. +- A link to the merge request that introduced the violation. +- The merge request's branch path in the format `[source] into [target]`. +- A list of users that committed changes to the merge request. +- A list of users that commented on the merge request. +- A list of users that approved the merge request. +- The user that merged the merge request. -## View the compliance report for a group +### View the compliance violations report for a group Prerequisites: - You must be an administrator or have the Owner role for the group. -To view the compliance report: +To view the compliance violations report: 1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Security & Compliance > Compliance report**. +1. On the left sidebar, select **Security and Compliance > Compliance report**. + +You can sort the compliance report on: -### Severity levels scale +- Severity level. +- Type of violation. +- Merge request title. -The following is a list of available violation severity levels, ranked from most to least severe: +Select a row to see details of the compliance violation. + +#### Severity levels + +Each compliance violation has one of the following severities. + +<!-- vale gitlab.SubstitutionWarning = NO --> | Icon | Severity level | |:----------------------------------------------|:---------------| @@ -46,75 +66,69 @@ The following is a list of available violation severity levels, ranked from most | **{severity-low, 18, gl-fill-orange-300}** | Low | | **{severity-info, 18, gl-fill-blue-400}** | Info | -### Violation types +<!-- vale gitlab.SubstitutionWarning = YES --> -The following is a list of violations that are either: +#### Violation types -- Already available. -- Aren't available, but which we are tracking in issues. +From [GitLab 14.10](https://gitlab.com/groups/gitlab-org/-/epics/6870), these are the available compliance violations. -| Violation | Severity level | Category | Description | Availability | -|:-------------------------------------|:----------------|:---------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------| -| Author approved merge request | High | [Separation of duties](#separation-of-duties) | The author of the merge request approved their own merge request. For more information, see [Prevent approval by author](../../project/merge_requests/approvals/settings.md#prevent-approval-by-author). | [Available in GitLab 14.10](https://gitlab.com/groups/gitlab-org/-/epics/6870) | -| Committers approved merge request | High | [Separation of duties](#separation-of-duties) | The committers of the merge request approved the merge request they contributed to. For more information, see [Prevent approvals by users who add commits](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). | [Available in GitLab 14.10](https://gitlab.com/groups/gitlab-org/-/epics/6870) | -| Fewer than two approvals | High | [Separation of duties](#separation-of-duties) | The merge request was merged with fewer than two approvals. For more information, see [Merge request approval rules](../../project/merge_requests/approvals/rules.md). | [Available in GitLab 14.10](https://gitlab.com/groups/gitlab-org/-/epics/6870) | -| Pipeline failed | Medium | [Pipeline results](../../../ci/pipelines/index.md) | The merge requests pipeline failed and was merged. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | -| Pipeline passed with warnings | Info | [Pipeline results](../../../ci/pipelines/index.md) | The merge request pipeline passed with warnings and was merged. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | -| Code coverage down more than 10% | High | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of more than 10%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | -| Code coverage down between 5% to 10% | Medium | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of between 5% to 10%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | -| Code coverage down between 1% to 5% | Low | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of between 1% to 5%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | -| Code coverage down less than 1% | Info | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of less than 1%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Violation | Severity level | Category | Description | +|:----------------------------------|:---------------|:----------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Author approved merge request | High | [Separation of duties](#separation-of-duties) | Author of the merge request approved their own merge request. For more information, see [Prevent approval by author](../../project/merge_requests/approvals/settings.md#prevent-approval-by-author). | +| Committers approved merge request | High | [Separation of duties](#separation-of-duties) | Committers of the merge request approved the merge request they contributed to. For more information, see [Prevent approvals by users who add commits](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). | +| Fewer than two approvals | High | [Separation of duties](#separation-of-duties) | Merge request was merged with fewer than two approvals. For more information, see [Merge request approval rules](../../project/merge_requests/approvals/rules.md). | -## Merge request drawer +The following are unavailable compliance violations that are tracked in [epic 5237](https://gitlab.com/groups/gitlab-org/-/epics/5237). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299357) in GitLab 14.1. +<!-- vale gitlab.SubstitutionWarning = NO --> -When you select a row, a drawer is shown that provides further details about the merge -request: +| Violation | Severity level | Category | Description | +|:-------------------------------------|:---------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------| +| Pipeline failed | Medium | [Pipeline results](../../../ci/pipelines/index.md) | Merge requests pipeline failed and was merged. | +| Pipeline passed with warnings | Info | [Pipeline results](../../../ci/pipelines/index.md) | Merge request pipeline passed with warnings and was merged. | +| Code coverage down more than 10% | High | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of more than 10%. | +| Code coverage down between 5% to 10% | Medium | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of between 5% to 10%. | +| Code coverage down between 1% to 5% | Low | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of between 1% to 5%. | +| Code coverage down less than 1% | Info | [Code coverage](../../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) | Code coverage report for the merge request indicates a reduction in coverage of less than 1%. | -- Project name and [compliance framework label](../../project/settings/index.md#add-a-compliance-framework-to-a-project), - if the project has one assigned. -- Link to the merge request. -- The merge request's branch path in the format `[source] into [target]`. -- A list of users that committed changes to the merge request. -- A list of users that commented on the merge request. -- A list of users that approved the merge request. -- The user that merged the merge request. +<!-- vale gitlab.SubstitutionWarning = YES --> -## Separation of duties +##### Separation of duties -We support a separation of duties policy between users who create and approve merge requests. -Our criteria for the separation of duties is as follows: +GitLab supports a separation of duties policy between users who create and approve merge requests. Our criteria for the +separation of duties is: -- [A merge request author is **not** allowed to approve their merge request](../../project/merge_requests/approvals/settings.md#prevent-approval-by-author) -- [A merge request committer is **not** allowed to approve a merge request they have added commits to](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits) -- [The minimum number of approvals required to merge a merge request is **at least** two](../../project/merge_requests/approvals/rules.md) +- [A merge request author is **not** allowed to approve their merge request](../../project/merge_requests/approvals/settings.md#prevent-approval-by-author). +- [A merge request committer is **not** allowed to approve a merge request they have added commits to](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). +- [The minimum number of approvals required to merge a merge request is **at least** two](../../project/merge_requests/approvals/rules.md). -## Chain of Custody report +### Chain of Custody report > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213364) in GitLab 13.3. > - Chain of Custody reports sent using email [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342594) in GitLab 15.3 with a flag named `async_chain_of_custody_report`. Disabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/370100) in GitLab 15.5. Feature flag `async_chain_of_custody_report` removed. > - Chain of Custody report includes all commits (instead of just merge commits) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267601) in GitLab 15.9 with a flag named `all_commits_compliance_report`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112092) in GitLab 15.9. Feature flag `all_commits_compliance_report` removed. -FLAG: -On self-managed GitLab, by default the Chain of Custody report only contains information on merge commits. To make the report contain information on all commits to projects within a group, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `all_commits_compliance_report`. On GitLab.com, this feature is not available. - -The Chain of Custody report allows customers to export a list of merge commits within the group. -The data provides a comprehensive view with respect to merge commits. It includes the merge commit SHA, -merge request author, merge request ID, merge user, date merged, pipeline ID, group name, project name, and merge request approvers. -Depending on the merge strategy, the merge commit SHA can be a merge commit, squash commit, or a diff head commit. - -With the `all_commits_compliance_report` flag enabled, the Chain of Custody report provides a 1 month trailing window of any commit into a project under the group. +The Chain of Custody report provides a 1 month trailing window of all commits to a project under the group. To generate the report for all commits, GitLab: 1. Fetches all projects under the group. -1. For each project, fetches the last 1 month of commits. Each project is capped at 1024 commits. If there are more than 1024 commits in the 1-month window, they - are truncated. -1. Writes the commits to a CSV file. The file is truncated at 15 MB because the report is emailed as an attachment. +1. For each project, fetches the last 1 month of commits. Each project is capped at 1024 commits. If there are more than + 1024 commits in the 1-month window, they are truncated. +1. Writes the commits to a CSV file. The file is truncated at 15 MB because the report is emailed as an attachment + (GitLab 15.5 and later). + +The report includes: + +- Commit SHA. +- Commit author. +- Committer. +- Date committed. +- Group. +- Project. -The expanded report includes the commit SHA, commit author, committer, date committed, the group, and the project. If the commit has a related merge commit, then the following are also included: - Merge commit SHA. @@ -124,39 +138,152 @@ If the commit has a related merge commit, then the following are also included: - Pipeline ID. - Merge request approvers. +#### Generate Chain of Custody report + To generate the Chain of Custody report: 1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Security & Compliance > Compliance report**. +1. On the left sidebar, select **Security and Compliance > Compliance report**. 1. Select **List of all merge commits**. -The Chain of Custody report is either: +Depending on your version of GitLab, the Chain of Custody report is either sent through email or available for download. + +#### Generate commit-specific Chain of Custody report + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267629) in GitLab 13.6. +> - Support for including all commits instead of only merge commits [added](https://gitlab.com/gitlab-org/gitlab/-/issues/393446) in GitLab 15.10. + +You can generate a commit-specific Chain of Custody report for a given commit SHA. This report provides only the +details for the provided commit SHA. + +To generate a commit-specific Chain of Custody report: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. At the top of the compliance report, to the right of **List of all commits**, select the down arrow + (**{chevron-lg-down}**). +1. Enter the commit SHA, and then select **Export commit custody report**. + +Depending on your version of GitLab, the Chain of Custody report is either sent through email or available for download. + +Alternatively, use a direct link: `https://gitlab.com/groups/<group-name>/-/security/merge_commit_reports.csv?commit_sha={optional_commit_sha}`, +passing in an optional value to the `commit_sha` query parameter. + +## Compliance frameworks report + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387910) in GitLab 15.10. -- Available for download. -- Sent through email. Requires GitLab 15.5 and later. +With compliance frameworks report, you can see the compliance frameworks that are applied to projects in a group. Each row of the report shows: -### Commit-specific Chain of Custody report +- Project name. +- Project path. +- Compliance framework label if the project has one assigned. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267629) in GitLab 13.6. +The default framework for the group has a **default** badge. -Authenticated group owners can generate a commit-specific Chain of Custody report for a given commit SHA, either: +### View the compliance frameworks report for a group -- Using the GitLab UI: +Prerequisites: + +- You must be an administrator or have the Owner role for the group. - 1. On the top bar, select **Main menu > Groups** and find your group. - 1. On the left sidebar, select **Security & Compliance > Compliance report**. - 1. At the top of the compliance report, to the right of **List of all merge commits**, select the down arrow (**{chevron-lg-down}**). - 1. Enter the merge commit SHA, and then select **Export commit custody report**. - SHA and then select **Export commit custody report**. +To view the compliance frameworks report: -The Chain of Custody report is either: +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. -- Available for download. -- Sent through email. Requires GitLab 15.5 and later. +### Apply a compliance framework to projects in a group -- Using a direct link: `https://gitlab.com/groups/<group-name>/-/security/merge_commit_reports.csv?commit_sha={optional_commit_sha}`, passing in an optional value to the - `commit_sha` query parameter. +> - Adding compliance frameworks using bulk actions [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383209) in GitLab 15.11. +> - Adding compliance frameworks without using bulk actions [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/394795) in GitLab 16.0. -NOTE: -The Chain of Custody report download is a CSV file, with a maximum size of 15 MB. -The remaining records are truncated when this limit is reached. +You can apply a compliance framework to projects in a group. + +Prerequisites: + +- You must have the Owner role for the group. + +To apply a compliance framework to one project in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Next to the project you want to add the compliance framework to, select **{plus}** **Add framework**. +1. Select an existing compliance framework or create a new one. + +To apply a compliance framework to multiple projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Select multiple projects. +1. From the **Choose one bulk action** dropdown list, select **Apply framework to selected projects**. +1. Select framework to apply. +1. Select **Apply**. + +### Remove a compliance framework from projects in a group + +> - Removing compliance frameworks using bulk actions [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383209) in GitLab 15.11. +> - Removing compliance frameworks without using bulk actions [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/394795) in GitLab 16.0. + +You can remove a compliance framework from projects in a group. + +Prerequisites: + +- You must have the Owner role for the group. + +To remove a compliance framework from one project in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Next to the compliance framework to remove from the project, select **{close}** on the framework label. + +To remove a compliance framework from multiple projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. Select multiple projects. +1. From the **Choose one bulk action** dropdown list, select **Remove framework from selected projects**. +1. Select **Remove**. + +### Export a report of compliance frameworks on projects in a group + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387912) in GitLab 16.0. + +Export a report of compliance frameworks that are applied to projects in a group. Reports: + +- Do not use filters on the framework report. +- Are truncated at 15 MB so the email attachment too large. + +Prerequisites: + +- You must be an administrator or have the Owner role for the group. + +To export a report of compliance frameworks on projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security and Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. On the Frameworks tab, select the **Export as CSV** action in the top right corner + +A report is compiled and delivered to your email inbox as an attachment. + +#### Filter the compliance frameworks report + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387911) in GitLab 15.11. + +To filter the list of compliance frameworks: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Compliance report**. +1. On the page, select the **Frameworks** tab. +1. In the search field: + 1. Select the attribute you want to filter by. + 1. Select an operator. + 1. Select from the list of options or enter text for the search. +1. Select **Search** (**{search}**). + +Repeat this process to filter by multiple attributes. diff --git a/doc/user/compliance/img/license-check_v13_4.png b/doc/user/compliance/img/license-check_v13_4.png Binary files differdeleted file mode 100644 index bc80f938395..00000000000 --- a/doc/user/compliance/img/license-check_v13_4.png +++ /dev/null diff --git a/doc/user/compliance/img/license_approval_policy_v15_9.png b/doc/user/compliance/img/license_approval_policy_v15_9.png Binary files differindex 43b1f89a07c..208643b47ae 100644 --- a/doc/user/compliance/img/license_approval_policy_v15_9.png +++ b/doc/user/compliance/img/license_approval_policy_v15_9.png diff --git a/doc/user/compliance/img/policies_maintainer_add_v14_3.png b/doc/user/compliance/img/policies_maintainer_add_v14_3.png Binary files differdeleted file mode 100644 index 7a27899f8c9..00000000000 --- a/doc/user/compliance/img/policies_maintainer_add_v14_3.png +++ /dev/null diff --git a/doc/user/compliance/img/policies_maintainer_edit_v14_3.png b/doc/user/compliance/img/policies_maintainer_edit_v14_3.png Binary files differdeleted file mode 100644 index 256c66bf7d8..00000000000 --- a/doc/user/compliance/img/policies_maintainer_edit_v14_3.png +++ /dev/null diff --git a/doc/user/compliance/img/policies_v13_0.png b/doc/user/compliance/img/policies_v13_0.png Binary files differdeleted file mode 100644 index 4918a0e6b62..00000000000 --- a/doc/user/compliance/img/policies_v13_0.png +++ /dev/null diff --git a/doc/user/compliance/index.md b/doc/user/compliance/index.md index 7981ab44bf9..ad36684d987 100644 --- a/doc/user/compliance/index.md +++ b/doc/user/compliance/index.md @@ -7,6 +7,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Compliance **(ULTIMATE)** -The compliance tools provided by GitLab help you keep an eye on various aspects of your project. For more information -on GitLab compliance features for projects, groups, and instances, see +The compliance tools provided by GitLab help you keep an eye on various aspects of your project, including: + +- [Compliance report](compliance_report/index.md). +- [License approval policies](license_approval_policies.md). +- [License list](license_list.md). +- [License scanning of CycloneDX files](license_scanning_of_cyclonedx_files/index.md). + +For more information on other GitLab compliance features for projects, groups, and instances, see [Compliance features](../../administration/compliance.md). diff --git a/doc/user/compliance/license_approval_policies.md b/doc/user/compliance/license_approval_policies.md index 32c90a1d317..860c2008021 100644 --- a/doc/user/compliance/license_approval_policies.md +++ b/doc/user/compliance/license_approval_policies.md @@ -5,11 +5,24 @@ group: Security Policies info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# License Approval Policies **(ULTIMATE)** +# License approval policies **(ULTIMATE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `license_scanning_policies`. Disabled by default. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8092) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `license_scanning_policies`. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/397644) in GitLab 15.11. Feature flag `license_scanning_policies` removed. -License Approval Policies allow you to specify multiple types of criteria that define when approval is required before a merge request can be merged in. +License approval policies allow you to specify multiple types of criteria that define when approval is required before a merge request can be merged in. + +NOTE: +License approval policies are applicable only to [protected](../project/protected_branches.md) target branches. + +The following video provides an overview of these policies. + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=34qBQ9t8qO8">Overview of GitLab License Approval Policies</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/34qBQ9t8qO8" frameborder="0" allowfullscreen> </iframe> +</figure> ## Create a new license approval policy diff --git a/doc/user/compliance/license_check_rules.md b/doc/user/compliance/license_check_rules.md index 968cf49ffdf..3007d04a8c1 100644 --- a/doc/user/compliance/license_check_rules.md +++ b/doc/user/compliance/license_check_rules.md @@ -1,84 +1,15 @@ --- -type: reference, howto -stage: Govern -group: Security Policies -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'license_approval_policies.md' +remove_date: '2023-07-25' --- -# License Check Policies (DEPRECATED) **(ULTIMATE)** +# License Check Policies (removed) **(ULTIMATE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/390417) in GitLab 15.9. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/390417) in GitLab 15.9 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/397067) in 16.0. +Use [License Approval Policies](license_approval_policies.md) instead. -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/390417) in GitLab 15.9. Users should migrate over to use [License Approval Policies](license_approval_policies.md) prior to GitLab 16.0. - -License check policies allow you to specify licenses that are `allowed` or `denied` in a project. If a `denied` -license is newly committed it blocks the merge request and instructs the developer to remove it. -Note, the merge request is not able to be merged until the `denied` license is removed. -You may add a [`License-Check` approval rule](#enabling-license-approvals-within-a-project), -which enables a designated approver that can approve and then merge a merge request with `denied` license. - -These policies can be configured by using the [Managed Licenses API](../../api/managed_licenses.md). - -![Merge request with denied licenses](img/denied_licenses_v15_3.png) - -The **Policies** tab in the project's license compliance section displays your project's license -policies. Project maintainers can specify policies in this section. - -![Edit Policy](img/policies_maintainer_edit_v14_3.png) - -![Add Policy](img/policies_maintainer_add_v14_3.png) - -Developers of the project can view the policies configured in a project. - -![View Policies](img/policies_v13_0.png) - -## Enabling License Approvals within a project - -Prerequisites: - -- Maintainer or Owner role. - -`License-Check` is a [merge request approval](../project/merge_requests/approvals/index.md) rule -you can enable to allow an individual or group to approve a merge request that contains a `denied` -license. - -You can enable `License-Check` one of two ways: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge request approvals**. -1. Select **Enable** or **Edit**. -1. Add or change the **Rule name** to `License-Check` (case sensitive). - -![License Check Approver Rule](img/license-check_v13_4.png) - -- Create an approval group in the [project policies section for License Compliance](license_check_rules.md#license-check-policies-deprecated). - You must set this approval group's number of approvals required to greater than zero. After you - enable this group in your project, the approval rule is enabled for all merge requests. - -Any code changes cause the approvals required to reset. - -An approval is required when a license report: - -- Contains a dependency that includes a software license that is `denied`. -- Is not generated during pipeline execution. - -An approval is optional when a license report: - -- Contains no software license violations. -- Contains only new licenses that are `allowed` or unknown. - -## Troubleshooting - -### The License Compliance widget is stuck in a loading state - -A loading spinner is displayed in the following scenarios: - -- While the pipeline is in progress. -- If the pipeline is complete, but still parsing the results in the background. -- If the license scanning job is complete, but the pipeline is still running. - -The License Compliance widget polls every few seconds for updated results. When the pipeline is complete, the first poll after pipeline completion triggers the parsing of the results. This can take a few seconds depending on the size of the generated report. - -The final state is when a successful pipeline run has been completed, parsed, and the licenses displayed in the widget. +<!-- This redirect file can be deleted after <2023-07-25>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 43e88e89c18..b7a68317fba 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -5,13 +5,13 @@ group: Composition Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# License compliance (DEPRECATED) **(ULTIMATE)** +# License Compliance (deprecated) **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5483) in GitLab 11.0. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. Users should migrate over to use the [new method of license scanning](../license_scanning_of_cyclonedx_files/index.md) prior to GitLab 16.0. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) in GitLab 15.9. You should instead migrate to use [License approval policies](../license_approval_policies.md) and the [new method of license scanning](../license_scanning_of_cyclonedx_files/index.md) prior to GitLab 16.1. If you're using [GitLab CI/CD](../../../ci/index.md), you can use License Compliance to search your project's dependencies for their licenses. You can then decide whether to allow or deny the use of @@ -25,8 +25,8 @@ For the job to activate, License Finder needs to find a compatible package defin GitLab checks the License Compliance report, compares the licenses between the source and target branches, and shows the information right on the merge request. Denied licenses are indicated by a `x` red icon next to them as well as new licenses that -need a decision from you. In addition, you can [manually allow or deny](../license_check_rules.md) licenses in your -project's license compliance policy section. If a denied license is detected in a new commit, +need a decision from you. In addition, you can [manually allow or deny](../license_approval_policies.md) licenses in your +project's security policies section. If a denied license is detected in a new commit, GitLab blocks any merge requests containing that commit and instructs the developer to remove the license. @@ -49,11 +49,43 @@ that you can later download and analyze. WARNING: License Compliance Scanning does not support run-time installation of compilers and interpreters. +## Enable License Compliance + +To enable License Compliance in your project's pipeline, either: + +- Enable [Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance) + (provided by [Auto DevOps](../../../topics/autodevops/index.md)). +- Include the [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) in your `.gitlab-ci.yml` file. + +License Compliance is not supported when GitLab is run with FIPS mode enabled. + +### Include the License Scanning template + +Prerequisites: + +- [GitLab Runner](../../../ci/runners/index.md) available, with the + [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). If you're using the + shared runners on GitLab.com, this is enabled by default. +- License Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the + `.gitlab-ci.yml` file, the `test` stage is required. +- [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) must be disabled. + +To [include](../../../ci/yaml/index.md#includetemplate) the +[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml), add it to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: Security/License-Scanning.gitlab-ci.yml +``` + +The included template creates a `license_scanning` job in your CI/CD pipeline and scans your +dependencies to find their licenses. + ## License expressions GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/). License compliance can read multiple licenses, but always considers them combined using the `AND` operator. For example, -if a dependency has two licenses, and one of them is allowed and the other is denied by the project [policy](../license_check_rules.md), +if a dependency has two licenses, and one of them is allowed and the other is denied by the project [license approval policy](../license_approval_policies.md), GitLab evaluates the composite license as _denied_, as this is the safer option. The ability to support other license expression operators (like `OR`, `WITH`) is tracked in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/6571). @@ -89,39 +121,7 @@ The reported licenses might be incomplete or inaccurate. | Rust | [Cargo](https://crates.io/) | | PHP | [Composer](https://getcomposer.org/) | -## Enable License Compliance - -To enable License Compliance in your project's pipeline, either: - -- Enable [Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance) - (provided by [Auto DevOps](../../../topics/autodevops/index.md)). -- Include the [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) in your `.gitlab-ci.yml` file. - -License Compliance is not supported when GitLab is run with FIPS mode enabled. - -### Include the License Scanning template - -Prerequisites: - -- [GitLab Runner](../../../ci/runners/index.md) available, with the - [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). If you're using the - shared runners on GitLab.com, this is enabled by default. -- License Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the - `.gitlab-ci.yml` file, the `test` stage is required. -- [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) must be disabled. - -To [include](../../../ci/yaml/index.md#includetemplate) the -[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml), add it to your `.gitlab-ci.yml` file: - -```yaml -include: - - template: Security/License-Scanning.gitlab-ci.yml -``` - -The included template creates a `license_scanning` job in your CI/CD pipeline and scans your -dependencies to find their licenses. - -### Available CI/CD variables +## Available CI/CD variables License Compliance can be configured using CI/CD variables. @@ -141,7 +141,7 @@ License Compliance can be configured using CI/CD variables. | `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address to download the analyzer from. | | `SETUP_CMD` | no | Custom setup for the dependency installation (experimental). | -### Installing custom dependencies +## Installing custom dependencies > Introduced in GitLab 11.4. @@ -169,7 +169,7 @@ variables: In this example, `my-custom-install-script.sh` is a shell script at the root directory of your project. -### Working with Monorepos +## Working with Monorepos Depending on your language, you may need to specify the path to the individual projects of a monorepo using the `LICENSE_FINDER_CLI_OPTS` variable. Passing in @@ -184,7 +184,7 @@ variables: LICENSE_FINDER_CLI_OPTS: "--aggregate_paths=relative-path/to/sub-project/one relative-path/to/sub-project/two" ``` -### Overriding the template +## Overriding the template WARNING: Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) @@ -203,7 +203,7 @@ license_scanning: CI_DEBUG_TRACE: "true" ``` -### Configuring Maven projects +## Configuring Maven projects The License Compliance tool provides a `MAVEN_CLI_OPTS` CI/CD variable which can hold the command line arguments to pass to the `mvn install` command which is executed under the hood. @@ -227,7 +227,7 @@ to explicitly add `-DskipTests` to your options. If you still need to run tests during `mvn install`, add `-DskipTests=false` to `MAVEN_CLI_OPTS`. -#### Using private Maven repositories +### Using private Maven repositories If you have a private Maven repository which requires login credentials, you can use the `MAVEN_CLI_OPTS` CI/CD variable. @@ -250,13 +250,13 @@ Alternatively, you can use a Java key store to verify the TLS connection. For in generate a key store file, see the [Maven Guide to Remote repository access through authenticated HTTPS](https://maven.apache.org/guides/mini/guide-repository-ssl.html). -### Selecting the version of Java +## Selecting the version of Java License Compliance uses Java 8 by default. You can specify a different Java version using `LM_JAVA_VERSION`. `LM_JAVA_VERSION` only accepts versions: 8, 11, 14, 15. -### Selecting the version of Python +## Selecting the version of Python > - [Introduced](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/36) in GitLab 12.0. > - In [GitLab 12.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12032), Python 3.5 became the default. @@ -277,22 +277,22 @@ license_scanning: `LM_PYTHON_VERSION` or `ASDF_PYTHON_VERSION` can be used to specify the desired version of Python. When both variables are specified `LM_PYTHON_VERSION` takes precedence. -### Custom root certificates for Python +## Custom root certificates for Python You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). -#### Using private Python repositories +### Using private Python repositories If you have a private Python repository you can use the `PIP_INDEX_URL` [CI/CD variable](#available-cicd-variables) to specify its location. -### Configuring npm projects +## Configuring npm projects You can configure npm projects by using an [`.npmrc`](https://docs.npmjs.com/configuring-npm/npmrc.html/) file. -#### Using private npm registries +### Using private npm registries If you have a private npm registry you can use the [`registry`](https://docs.npmjs.com/using-npm/config/#registry) @@ -304,7 +304,7 @@ For example: registry = https://npm.example.com ``` -#### Custom root certificates for npm +### Custom root certificates for npm You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). @@ -318,12 +318,12 @@ For example: strict-ssl = false ``` -### Configuring Yarn projects +## Configuring Yarn projects You can configure Yarn projects by using a [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc/) file. -#### Using private Yarn registries +### Using private Yarn registries If you have a private Yarn registry you can use the [`npmRegistryServer`](https://yarnpkg.com/configuration/yarnrc/#npmRegistryServer) @@ -335,17 +335,17 @@ For example: npmRegistryServer: "https://npm.example.com" ``` -#### Custom root certificates for Yarn +### Custom root certificates for Yarn You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). -### Configuring Bower projects +## Configuring Bower projects You can configure Bower projects by using a [`.bowerrc`](https://bower.io/docs/config/#bowerrc-specification) file. -#### Using private Bower registries +### Using private Bower registries If you have a private Bower registry you can use the [`registry`](https://bower.io/docs/config/#bowerrc-specification) @@ -359,16 +359,16 @@ For example: } ``` -#### Custom root certificates for Bower +### Custom root certificates for Bower You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by specifying a `ca` setting in a [`.bowerrc`](https://bower.io/docs/config/#bowerrc-specification) file. -### Configuring Bundler projects +## Configuring Bundler projects -#### Using private Bundler registries +### Using private Bundler registries If you have a private Bundler registry you can use the [`source`](https://bundler.io/man/gemfile.5.html#GLOBAL-SOURCES) @@ -380,7 +380,7 @@ For example: source "https://gems.example.com" ``` -#### Custom root certificates for Bundler +### Custom root certificates for Bundler You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by @@ -388,9 +388,9 @@ specifying a [`BUNDLE_SSL_CA_CERT`](https://bundler.io/v2.0/man/bundle-config.1. [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Cargo projects +## Configuring Cargo projects -#### Using private Cargo registries +### Using private Cargo registries If you have a private Cargo registry you can use the [`registries`](https://doc.rust-lang.org/cargo/reference/registries.html) @@ -403,7 +403,7 @@ For example: my-registry = { index = "https://my-intranet:8080/git/index" } ``` -#### Custom root certificates for Cargo +### Custom root certificates for Cargo To supply a custom root certificate to complete TLS verification, do one of the following: @@ -412,9 +412,9 @@ To supply a custom root certificate to complete TLS verification, do one of the [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Composer projects +## Configuring Composer projects -#### Using private Composer registries +### Using private Composer registries If you have a private Composer registry you can use the [`repositories`](https://getcomposer.org/doc/05-repositories.md) @@ -437,7 +437,7 @@ For example: } ``` -#### Custom root certificates for Composer +### Custom root certificates for Composer You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by @@ -445,7 +445,7 @@ specifying a [`COMPOSER_CAFILE`](https://getcomposer.org/doc/03-cli.md#composer- [variable](../../../ci/variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) in the job definition. -### Configuring Conan projects +## Configuring Conan projects You can configure [Conan](https://conan.io/) projects by adding a `.conan` directory to your project root. The project root serves as the [`CONAN_USER_HOME`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-user-home). @@ -474,7 +474,7 @@ NOTE: `license_scanning` image ships with [Mono](https://www.mono-project.com/) and [MSBuild](https://github.com/mono/msbuild#microsoftbuild-msbuild). Additional setup may be required to build packages for this project configuration. -#### Using private Conan registries +### Using private Conan registries By default, [Conan](https://conan.io/) uses the `conan-center` remote. For example: @@ -508,7 +508,7 @@ example: If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/index.md#protect-a-cicd-variable) following the naming convention described in the [`CONAN_LOGIN_USERNAME` documentation](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name). -#### Custom root certificates for Conan +### Custom root certificates for Conan You can provide custom certificates by adding a `.conan/cacert.pem` file to the project root and setting [`CA_CERT_PATH`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-cacert-path) @@ -518,7 +518,7 @@ If you specify the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd- variable's X.509 certificates are installed in the Docker image's default trust store and Conan is configured to use this as the default `CA_CERT_PATH`. -### Configuring Go projects +## Configuring Go projects To configure [Go modules](https://github.com/golang/go/wiki/Modules) based projects, specify [CI/CD variables](https://pkg.go.dev/cmd/go#hdr-Environment_variables) @@ -528,14 +528,14 @@ If a project has [vendored](https://pkg.go.dev/cmd/go#hdr-Vendor_Directories) it then the combination of the `vendor` directory and `mod.sum` file are used to detect the software licenses associated with the Go module dependencies. -#### Using private Go registries +### Using private Go registries You can use the [`GOPRIVATE`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) and [`GOPROXY`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) environment variables to control where modules are sourced from. Alternatively, you can use [`go mod vendor`](https://go.dev/ref/mod#tmp_28) to vendor a project's modules. -#### Custom root certificates for Go +### Custom root certificates for Go You can specify the [`-insecure`](https://pkg.go.dev/cmd/go/internal/get) flag by exporting the [`GOFLAGS`](https://pkg.go.dev/cmd/go#hdr-Environment_variables) @@ -550,7 +550,7 @@ license_scanning: GOFLAGS: '-insecure' ``` -#### Using private NuGet registries +### Using private NuGet registries If you have a private NuGet registry you can add it as a source by adding it to the [`packageSources`](https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file#package-source-sections) @@ -568,7 +568,7 @@ For example: </configuration> ``` -#### Custom root certificates for NuGet +### Custom root certificates for NuGet You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). @@ -654,7 +654,7 @@ registry.gitlab.com/security-products/license-finder:latest The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../../application_security/index.md#vulnerability-scanner-maintenance) +process by which external resources can be imported or temporarily accessed. These scanners are [updated periodically](../../application_security/index.md#vulnerability-scanner-maintenance) with new definitions, so consider if you are able to make periodic updates yourself. For details on saving and transporting Docker images as a file, see the Docker documentation on @@ -693,8 +693,8 @@ Additional configuration may be needed for connecting to private registries for: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212388) in GitLab 13.3. -Prior to GitLab 13.3, offline environments required an exact name match for [project policies](../license_check_rules.md). -In GitLab 13.3 and later, GitLab matches the name of [project policies](../license_check_rules.md) +Prior to GitLab 13.3, offline environments required an exact name match for [project policies](../license_approval_policies.md). +In GitLab 13.3 and later, GitLab matches the name of [project policies](../license_approval_policies.md) with identifiers from the [SPDX license list](https://spdx.org/licenses/). A local copy of the SPDX license list is distributed with the GitLab instance. If needed, the GitLab instance's administrator can manually update it with a [Rake task](../../../raketasks/spdx.md). diff --git a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md index 483c15d648c..832f1007a91 100644 --- a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md +++ b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md @@ -7,25 +7,28 @@ info: To determine the technical writer assigned to the Stage/Group associated w # License scanning of CycloneDX files **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384932) in GitLab 15.9 [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`. Both flags are disabled by default and both flags must be enabled for this feature to work. - -FLAG: -On self-managed GitLab, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384932) in GitLab 15.9 for GitLab SaaS [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`. Both flags are disabled by default and both flags must be enabled for this feature to work. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.10 for GitLab SaaS. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.10 for self-managed GitLab [with two flags](../../../administration/feature_flags.md) named `license_scanning_sbom_scanner` and `package_metadata_synchronization`, both of which must be enabled for this feature to work. The flags are disabled by default due to the initial performance load when the license database is first imported. Work to improve performance is being tracked in [issue 397670](https://gitlab.com/gitlab-org/gitlab/-/issues/397670). +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/385173) in GitLab 15.11 for self-managed GitLab. To detect the licenses in use, License Compliance relies on running the [Dependency Scanning CI Jobs](../../application_security/dependency_scanning/index.md), and analyzing the [CycloneDX](https://cyclonedx.org/) Software Bill of Materials (SBOM) generated by those jobs. Other 3rd party scanners may also be used as long as they produce a CycloneDX file with a list of dependencies for [one of our supported languages](#supported-languages-and-package-managers). -This method of scanning is also capable of parsing and identifying over 500 different types of licenses -and can extract license information from packages that are dual-licensed or have multiple different licenses that apply. +This method of scanning is also capable of parsing and identifying over 500 different types of licenses, as defined in [the SPDX list](https://spdx.org/licenses/). +Licenses not in the SPDX list are reported as "Unknown". License information can also be extracted from packages that are dual-licensed, or have multiple different licenses that apply. + +## Enable license scanning -To enable license detection using Dependency Scanning in a project, -include the `Jobs/Dependency-Scanning.yml` template in its CI configuration, -but do not include the `Jobs/License-Scanning.yml` template. +Prerequisites: -## Requirements +- Enable [Synchronization with the GitLab License Database](../../admin_area/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in Admin Area for the GitLab instance. +- Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration) + and ensure that its prerequisites are met. -The license scanning requirements are the same as those for [Dependency Scanning](../../application_security/dependency_scanning/index.md#requirements). +From the `.gitlab-ci.yml` file, remove the deprecated line `Jobs/License-Scanning.gitlab-ci.yml`, if +it's present. ## Supported languages and package managers @@ -66,10 +69,13 @@ License scanning is supported for the following languages and package managers: <td><a href="https://maven.apache.org/">Maven</a></td> </tr> <tr> - <td rowspan="2">JavaScript and TypeScript</td> + <td rowspan="3">JavaScript and TypeScript</td> <td><a href="https://www.npmjs.com/">npm</a></td> </tr> <tr> + <td><a href="https://pnpm.io/">pnpm</a></td> + </tr> + <tr> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> </tr> <tr> @@ -104,11 +110,6 @@ License scanning is supported for the following languages and package managers: The supported files and versions are the ones supported by [Dependency Scanning](../../application_security/dependency_scanning/index.md#supported-languages-and-package-managers). -## Configuration - -To enable license scanning of CycloneDX files, -you must configure [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration). - ## License expressions GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/). @@ -121,3 +122,23 @@ in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/6571). ## Blocking merge requests based on detected licenses Users can require approval for merge requests based on the licenses that are detected by configuring a [license approval policy](../license_approval_policies.md). + +## Running in an offline environment + +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required to successfully scan +CycloneDX reports for licenses. For more information, see the offline [quick start guide](../../../topics/offline/quick_start_guide.md#enabling-the-package-metadata-database). + +## Troubleshooting + +### A CycloneDX file is not being scanned and appears to provide no results + +Ensure that the CycloneDX file adheres to the [CycloneDX JSON specification](https://cyclonedx.org/docs/latest/json). This specification does [not permit duplicate entries](https://cyclonedx.org/docs/latest/json/#components). Projects that contain multiple SBOM files should either report each SBOM file up as individual CI report artifacts or they should ensure that duplicates are removed if the SBOMs are merged as part of the CI pipeline. + +You can validate CycloneDX SBOM files against the `CycloneDX JSON specification` as follows: + +```shell +$ docker run -it --rm -v "$PWD:/my-cyclonedx-sboms" -w /my-cyclonedx-sboms cyclonedx/cyclonedx-cli:latest cyclonedx validate --input-version v1_4 --input-file gl-sbom-all.cdx.json + +Validating JSON BOM... +BOM validated successfully. +``` diff --git a/doc/user/crm/index.md b/doc/user/crm/index.md index ebacda506b4..54e87118361 100644 --- a/doc/user/crm/index.md +++ b/doc/user/crm/index.md @@ -1,6 +1,6 @@ --- -stage: Plan -group: Certify +stage: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/discussions/img/index_notes_filters.png b/doc/user/discussions/img/index_notes_filters.png Binary files differdeleted file mode 100644 index 977a3770c05..00000000000 --- a/doc/user/discussions/img/index_notes_filters.png +++ /dev/null diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 9c9b5301460..096b4dc2cc5 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -36,6 +36,8 @@ You can create comments in places like: - Issues - Merge requests - Snippets +- Tasks +- OKRs Each object can have as many as 5,000 comments. @@ -46,12 +48,12 @@ instance with `@username` or `@groupname`. All mentioned users are notified with Users can change this setting for themselves in the [notification settings](../profile/notifications.md). You can quickly see which comments involve you, because -mentions for yourself (the user currently signed in) are highlighted +mentions for yourself (the user who is signed in) are highlighted in a different color. Avoid mentioning `@all` in issues and merge requests. It sends an email notification -to all members of that project's parent group, not only the participants of the project, -and may be interpreted as spam. +to all members of that project's parent group, not only the participants of the project. +It might be interpreted as spam. Notifications and mentions can be disabled in [a group's settings](../group/manage.md#disable-email-notifications). @@ -89,8 +91,8 @@ The comment is not displayed on your project's **Repository > Commits** page. NOTE: When your comment contains a reference to a commit included in the merge request, -it's automatically converted to a link in the context of the current merge request. -For example, `28719b171a056960dfdc0012b625d0b47b123196` becomes +it's converted to a link in the context of the merge request. +For example, `28719b171a056960dfdc0012b625d0b47b123196` becomes `28719b17` that links to `https://gitlab.example.com/example-group/example-project/-/merge_requests/12345/diffs?commit_id=28719b171a056960dfdc0012b625d0b47b123196`. ## Add a comment to a commit @@ -203,41 +205,35 @@ You can also mark an [issue as confidential](../project/issues/confidential_issu ## Show only comments -For issues and merge requests with many comments, you can filter the page to show comments only. +In discussions with many comments, filter the discussion to show only comments or history of +changes (system notes). System notes include changes to the description, mentions in other GitLab +objects, or changes to labels, assignees, and the milestone. +GitLab saves your preference, and applies it to every issue, merge request, or epic you view. 1. Open the **Overview** tab in a merge request, issue, or epic. -1. On the right side of the page, select from the filter: +1. On the right side of the page, from the **Sort or filter** dropdown list, select a filter: - **Show all activity**: Display all user comments and system notes. - (issue updates, mentions from other issues, changes to the description, and so on). - **Show comments only**: Display only user comments. - **Show history only**: Display only activity notes. -![Notes filters dropdown list options](img/index_notes_filters.png) +## Change activity sort order -GitLab saves your preference, so it persists when you visit the same page again -from any device you're logged into. +Reverse the default order and interact with the activity feed sorted by most recent items +at the top. GitLab saves your preference in local storage and applies it to every issue, +merge request, or epic you view. -## View description change history **(PREMIUM)** +To change the activity sort order: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10103) in GitLab 12.6. +1. Open the **Overview** tab in a merge request, issue, or epic. +1. On the right side of the page, from the **Sort or filter** dropdown list, select the sort order + **Newest first** or **Oldest first** (default). + +## View description change history **(PREMIUM)** You can see changes to the description listed in the history. To compare the changes, select **Compare with previous version**. -## Change activity sort order - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14588) in GitLab 12.10. - -You can reverse the default order and interact with the activity feed sorted by most recent items -at the top. Your preference is saved in local storage and automatically applies to every issue, -merge request, or epic you view. - -To change the activity sort order: - -1. Select the **Oldest first** (or **Newest first**) dropdown list. -1. Select either oldest or newest items to be shown first. - ## Assign an issue to the commenting user > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/191455) in GitLab 13.1. @@ -260,7 +256,7 @@ Prerequisites: To create a thread by replying to a comment: -1. In the upper right of the comment, select **Reply to comment** (**{comment}**). +1. In the upper-right corner of the comment, select **Reply to comment** (**{comment}**). ![Reply to comment button](img/reply_to_comment_button.png) @@ -283,7 +279,7 @@ Prerequisites: To create a thread: 1. Enter a comment. -1. Below the comment, to the right of the **Comment** button, select the down arrow (**{chevron-down}**). +1. Below the comment, to the right of **Comment**, select the down arrow (**{chevron-down}**). 1. From the list, select **Start thread**. 1. Select **Start thread** again. @@ -308,7 +304,7 @@ To resolve a thread: 1. Go to the thread. 1. Do one of the following: - - In the upper right of the original comment, select **Resolve thread** (**{check-circle}**). + - In the upper-right corner of the original comment, select **Resolve thread** (**{check-circle}**). - Below the last reply, in the **Reply** field, select **Resolve thread**. - Below the last reply, in the **Reply** field, enter text, select the **Resolve thread** checkbox, and select **Add comment now**. @@ -320,7 +316,7 @@ At the top of the page, the number of unresolved threads is updated: If you have multiple unresolved threads in a merge request, you can create an issue to resolve them separately. In the merge request, at the top of the page, -select the ellipsis icon button (**{ellipsis_v}**) in the threads control and then select **Create issue to resolve all threads**: +select the ellipsis icon button (**{ellipsis_v}**) in the threads control and then select **Resolve all with new issue**: ![Open new issue for all unresolved threads](img/create_new_issue_v15_4.png) diff --git a/doc/user/enterprise_user/index.md b/doc/user/enterprise_user/index.md index 3daee460956..ffeaaa8c591 100644 --- a/doc/user/enterprise_user/index.md +++ b/doc/user/enterprise_user/index.md @@ -22,7 +22,7 @@ A user account is considered an enterprise account when: - [SCIM](../group/saml_sso/scim_setup.md) creates the user account on behalf of the group. -A user can also [manually connect an identity provider (IdP) to a GitLab account whose email address matches the subscribing organization's domain](../group/saml_sso/index.md#linking-saml-to-your-existing-gitlabcom-account). +A user can also [manually connect an identity provider (IdP) to a GitLab account whose email address matches the subscribing organization's domain](../group/saml_sso/index.md#link-saml-to-your-existing-gitlabcom-account). By selecting **Authorize** when connecting these two accounts, the user account with the matching email address is classified as an enterprise user. However, this user account does not have an **Enterprise** badge in GitLab. @@ -31,6 +31,86 @@ Although a user can be a member of more than one group, each user account can be provisioned by only one group. As a result, a user is considered an enterprise user under one top-level group only. +## Verified domains for groups + +The following automated processes use [verified domains](../project/pages/custom_domains_ssl_tls_certification/index.md) to run: + +- [Bypass email confirmation for provisioned users](#bypass-email-confirmation-for-provisioned-users). + +### Set up a verified domain + +Prerequisites: + +- A custom domain name `example.com` or subdomain `subdomain.example.com`. +- Access to your domain's server control panel to set up a DNS `TXT` record to verify your domain's ownership. + +Setting up a verified domain is similar to [setting up a custom domain on GitLab Pages](../project/pages/custom_domains_ssl_tls_certification/index.md). However, you must: + +- Link the domain to a project. For more information on group-level domain verification, see [issue 5299](https://gitlab.com/groups/gitlab-org/-/epics/5299). +- Configure the DNS `TXT` record to verify the domain's ownership. + +Steps: + +#### 1. Add a custom domain for the matching email domain + +The custom domain must match the email domain exactly. For example, if your email is `username@example.com`, verify the `example.com` domain. + +1. On the top bar, select **Main menu > Groups** and find your top group. +1. On the left sidebar, select **Settings > Domain Verification**. +1. In the upper-right corner, select **Add Domain**. +1. In **Domain**, enter the domain name. +1. In **Project**, link to a project. +1. Optional. In **Certificate**, switch the **Manually enter certificate information** toggle to add an SSL/TLS + certificate. You can also add the certificate and key later. +1. Select **Add Domain**. + +#### 2. Get a verification code + +After you create a new domain, the verification code prompts you. Copy the values from GitLab +and paste them in your domain's control panel as a `TXT` record. + +![Get the verification code](../img/get_domain_verification_code_v16_0.png) + +#### 3. Verify the domain's ownership + +After you have added all the DNS records: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Domain Verification**. +1. On the domain table row, Select **Retry verification** (**{retry}**). + +![Verify your domain](../img/retry_domain_verification_v16_0.png) + +WARNING: +For GitLab instances with domain verification enabled, +if the domain cannot be verified for 7 days, that domain is removed +from the GitLab project. + +> **Notes:** +> +> - Domain verification is **required for GitLab.com users**; + for GitLab self-managed instances, your GitLab administrator has the option + to [disabled custom domain verification](../../administration/pages/index.md#custom-domain-verification). +> - [DNS propagation may take some time (up to 24 hours)](https://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/complete-guide-to-dns-records/), + although it's usually a matter of minutes to complete. Until it does, verification + fails, and attempts to visit your domain result in a 404. +> - Once your domain has been verified, leave the verification record + in place. Your domain is periodically reverified, and may be + disabled if the record is removed. + +### View domains in group + +To view all configured domains in your group: + +1. On the top bar, select **Main menu > Groups** and find your top-level group. +1. On the left sidebar, select **Settings > Domain Verification**. + +You then see: + +- A list of added domains. +- The domains' status of **Verified** or **Unverified**. +- The project where the domain has been configured. + ## Manage enterprise users in a namespace A top-level Owner of a namespace on a paid plan can retrieve information about and diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index fb32c64f06c..fae45e4b2d3 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -65,7 +65,7 @@ The IP addresses for `mg.gitlab.com` are subject to change at any time. On GitLab.com, there's a mailbox configured for Service Desk with the email address: `contact-project+%{key}@incoming.gitlab.com`. To use this mailbox, configure the -[custom suffix](../project/service_desk.md#configuring-a-custom-email-address-suffix) in project +[custom suffix](../project/service_desk.md#configure-a-custom-email-address-suffix) in project settings. ## Backups @@ -89,15 +89,21 @@ Similarly, you can clone a project's wiki to back it up. All files [uploaded after August 22, 2020](../project/wiki/index.md#create-a-new-wiki-page) are included when cloning. +## Delayed group deletion **(PREMIUM SAAS)** + +After May 08, 2023, all groups have delayed deletion enabled by default. + +Groups are permanently deleted after a seven-day delay. + +If you are on the Free tier, your groups are immediately deleted, and you will not be able to restore them. + ## Delayed project deletion **(PREMIUM SAAS)** -Top-level groups created after August 12, 2021 have delayed project deletion enabled by default. -Projects are permanently deleted after a seven-day delay. +After May 08, 2023, all groups have delayed project deletion enabled by default. -If you are on: +Projects are permanently deleted after a seven-day delay. -- Premium tier and above, you can disable this by changing the [group setting](../group/manage.md#enable-delayed-project-deletion). -- Free tier, you cannot disable this setting or restore projects. +If you are on the Free tier, your projects are immediately deleted, and you will not be able to restore them. ## Inactive project deletion @@ -125,20 +131,22 @@ Host gitlab.com ## GitLab Pages -Below are the settings for [GitLab Pages](https://about.gitlab.com/stages-devops-lifecycle/pages/). +Some settings for [GitLab Pages](../project/pages/index.md) differ from the +[defaults for self-managed instances](../../administration/pages/index.md): -| Setting | GitLab.com | Default | -|---------------------------|------------------------|------------------------| -| Domain name | `gitlab.io` | - | -| IP address | `35.185.44.232` | - | -| Custom domains support | **{check-circle}** Yes | **{dotted-circle}** No | -| TLS certificates support | **{check-circle}** Yes | **{dotted-circle}** No | -| [Maximum size](../../administration/pages/index.md#set-global-maximum-size-of-each-gitlab-pages-site) (compressed) | 1 GB | 100 MB | +| Setting | GitLab.com | +|:--------------------------------------------------|:-----------------------| +| Domain name | `gitlab.io` | +| IP address | `35.185.44.232` | +| Support for custom domains | **{check-circle}** Yes | +| Support for TLS certificates | **{check-circle}** Yes | +| Maximum site size | 1 GB | +| Number of custom domains per GitLab Pages website | 150 | -The maximum size of your Pages site is also regulated by the artifacts maximum size, +The maximum size of your Pages site depends on the maximum artifact size, which is part of [GitLab CI/CD](#gitlab-cicd). -There are also [rate limits set for GitLab Pages](#gitlabcom-specific-rate-limits). +[Rate limits](#gitlabcom-specific-rate-limits) also exist for GitLab Pages. ## GitLab CI/CD @@ -151,7 +159,7 @@ the related documentation. | Artifacts maximum size (compressed) | 1 GB | See [Maximum artifacts size](../../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size). | | Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | See [Default artifacts expiration](../admin_area/settings/continuous_integration.md#default-artifacts-expiration). | | Scheduled Pipeline Cron | `*/5 * * * *` | See [Pipeline schedules advanced configuration](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency). | -| Maximum jobs in active pipelines | `500` for Free tier, `1000` for all trial tiers, and unlimited otherwise. | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). | +| Maximum jobs in active pipelines | `500` for Free tier, `1000` for all trial tiers, `20000` for Premium, and `100000` for Ultimate. | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). | | Maximum CI/CD subscriptions to a project | `2` | See [Number of CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). | | Maximum number of pipeline triggers in a project | `25000` for Free tier, Unlimited for all paid tiers | See [Limit the number of pipeline triggers](../../administration/instance_limits.md#limit-the-number-of-pipeline-triggers). | | Maximum pipeline schedules in projects | `10` for Free tier, `50` for all paid tiers | See [Number of pipeline schedules](../../administration/instance_limits.md#number-of-pipeline-schedules). | @@ -175,7 +183,7 @@ varies by format: | Generic | 5 GB | | Helm | 5 MB | | Maven | 5 GB | -| npm: | 5 GB | +| npm | 5 GB | | NuGet | 5 GB | | PyPI | 5 GB | | Terraform | 1 GB | @@ -188,7 +196,7 @@ the default value [is the same as for self-managed instances](../admin_area/sett | Setting | GitLab.com default | |-------------------------------|--------------------| | [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | -| [Maximum import size](../project/settings/import_export.md#maximum-import-file-size) | 5 GB | +| [Maximum import size](../project/settings/import_export.md#import-a-project-and-its-data) | 5 GB | | Maximum attachment size | 100 MB | If you are near or over the repository size limit, you can either @@ -200,6 +208,30 @@ NOTE: Cloudflare. Git LFS and imports other than a file upload are not affected by this limit. Repository limits apply to both public and private projects. +## Default import sources + +> Disabling all importers by default for new GitLab self-managed installations [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118970) in GitLab 16.0. + +The import sources that are available by default depend on which GitLab you use: + +- GitLab.com: all available import sources are enabled by default. +- GitLab self-managed: no import sources are enabled by default and must be + [enabled](../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). + +| Import source | GitLab.com default | GitLab self-managed default | +|:----------------------------------------------------------------------------------------------------|:-----------------------|:----------------------------| +| [Bitbucket Cloud](../project/import/bitbucket.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [Bitbucket Server](../project/import/bitbucket_server.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [FogBugz](../project/import/fogbugz.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [Gitea](../project/import/gitea.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [GitLab by direct transfer](../group/import/index.md#migrate-groups-by-direct-transfer-recommended) | **{check-circle}** Yes | **{dotted-circle}** No | +| [GitLab using file exports](../project/settings/import_export.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [GitHub](../project/import/github.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [Manifest file](../project/import/manifest.md) | **{check-circle}** Yes | **{dotted-circle}** No | +| [Repository by URL](../project/import/repo_by_url.md) | **{check-circle}** Yes | **{dotted-circle}** No | + +[Other importers](../project/import/index.md#available-project-importers) are available. + ## IP range GitLab.com uses the IP ranges `34.74.90.64/28` and `34.74.226.0/24` for traffic from its Web/API @@ -247,7 +279,7 @@ The limit varies depending on your plan and the number of seats in your subscrip | Setting | Default for GitLab.com | |----------------------|-------------------------| -| Number of webhooks | `100` per project, `50` per group | +| Number of webhooks | `100` per project, `50` per group (subgroup webhooks are not counted towards parent group limits ) | | Maximum payload size | 25 MB | | Timeout | 10 seconds | @@ -263,73 +295,6 @@ Runner SaaS is the hosted, secure, and managed build environment you can use to For more information, see [Runner SaaS](../../ci/runners/index.md). -## Sidekiq - -GitLab.com runs [Sidekiq](https://sidekiq.org) with arguments `--timeout=4 --concurrency=4` -and the following environment variables: - -| Setting | GitLab.com | Default | -|----------------------------------------|------------|-----------| -| `GITLAB_MEMORY_WATCHDOG_ENABLED` | - | `true` | -| `SIDEKIQ_MEMORY_KILLER_MAX_RSS` | - | `2000000` | -| `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS` | - | - | -| `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL` | - | `3` | -| `SIDEKIQ_MEMORY_KILLER_GRACE_TIME` | - | `900` | -| `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT` | - | `30` | -| `SIDEKIQ_LOG_ARGUMENTS` | `1` | `1` | - -For more information, see how to -[configure the environment variables](../../administration/sidekiq/sidekiq_memory_killer.md). - -NOTE: -The `SIDEKIQ_MEMORY_KILLER_MAX_RSS` setting is `16000000` on Sidekiq import -nodes and Sidekiq export nodes. - -## PostgreSQL - -GitLab.com being a fairly large installation of GitLab means we have changed -various PostgreSQL settings to better suit our needs. For example, we use -streaming replication and servers in hot-standby mode to balance queries across -different database servers. - -The list of GitLab.com specific settings (and their defaults) is as follows: - -| Setting | GitLab.com | Default | -|:--------------------------------------|:--------------------------------------------------------------------|:--------------------------------------| -| `archive_command` | `/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-push %p` | empty | -| `archive_mode` | on | off | -| `autovacuum_analyze_scale_factor` | 0.01 | 0.01 | -| `autovacuum_max_workers` | 6 | 3 | -| `autovacuum_vacuum_cost_limit` | 1000 | -1 | -| `autovacuum_vacuum_scale_factor` | 0.01 | 0.02 | -| `checkpoint_completion_target` | 0.7 | 0.9 | -| `checkpoint_segments` | 32 | 10 | -| `effective_cache_size` | 338688 MB | Based on how much memory is available | -| `hot_standby` | on | off | -| `hot_standby_feedback` | on | off | -| `log_autovacuum_min_duration` | 0 | -1 | -| `log_checkpoints` | on | off | -| `log_line_prefix` | `%t [%p]: [%l-1]` | empty | -| `log_min_duration_statement` | 1000 | -1 | -| `log_temp_files` | 0 | -1 | -| `maintenance_work_mem` | 2048 MB | 16 MB | -| `max_replication_slots` | 5 | 0 | -| `max_wal_senders` | 32 | 0 | -| `max_wal_size` | 5 GB | 1 GB | -| `shared_buffers` | 112896 MB | Based on how much memory is available | -| `shared_preload_libraries` | `pg_stat_statements` | empty | -| `shmall` | 30146560 | Based on the server's capabilities | -| `shmmax` | 123480309760 | Based on the server's capabilities | -| `wal_buffers` | 16 MB | -1 | -| `wal_keep_segments` | 512 | 10 | -| `wal_level` | replica | minimal | -| `statement_timeout` | 15 s | 60 s | -| `idle_in_transaction_session_timeout` | 60 s | 60 s | - -Some of these settings are in the process being adjusted. For example, the value -for `shared_buffers` is quite high, and we are -[considering adjusting it](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4985). - ## Puma GitLab.com uses the default of 60 seconds for [Puma request timeouts](../../administration/operations/puma.md#change-the-worker-timeout). @@ -358,21 +323,23 @@ are also informational headers with this response detailed in The following table describes the rate limits for GitLab.com, both before and after the limits change in January, 2021: -| Rate limit | From 2021-02-12 | From 2022-02-03 | -|:---------------------------------------------------------------------------|:------------------------------|:----------------------------------------| -| **Protected paths** (for a given **IP address**) | **10** requests per minute | **10** requests per minute | -| **Raw endpoint** traffic (for a given **project, commit, and file path**) | **300** requests per minute | **300** requests per minute | -| **Unauthenticated** traffic (from a given **IP address**) | **500** requests per minute | **500** requests per minute | -| **Authenticated** API traffic (for a given **user**) | **2,000** requests per minute | **2,000** requests per minute | -| **Authenticated** non-API HTTP traffic (for a given **user**) | **1,000** requests per minute | **1,000** requests per minute | -| **All** traffic (from a given **IP address**) | **2,000** requests per minute | **2,000** requests per minute | -| **Issue creation** | **300** requests per minute | **200** requests per minute | -| **Note creation** (on issues and merge requests) | **60** requests per minute | **60** requests per minute | -| **Advanced, project, and group search** API (for a given **IP address**) | **10** requests per minute | **10** requests per minute | -| **GitLab Pages** requests (for a given **IP address**) | | **1000** requests per **50 seconds** | -| **GitLab Pages** requests (for a given **GitLab Pages domain**) | | **5000** requests per **10 seconds** | -| **Pipeline creation** requests (for a given **project, user, and commit**) | | **25** requests per minute | -| **Alert integration endpoint** requests (for a given **project**) | | **3600** requests per hour | +| Rate limit | From 2021-02-12 | From 2022-02-03 | +|:---------------------------------------------------------------------------|:------------------------------|:-------------------------------------| +| **Protected paths** (for a given **IP address**) | **10** requests per minute | **10** requests per minute | +| **Raw endpoint** traffic (for a given **project, commit, and file path**) | **300** requests per minute | **300** requests per minute | +| **Unauthenticated** traffic (from a given **IP address**) | **500** requests per minute | **500** requests per minute | +| **Authenticated** API traffic (for a given **user**) | **2,000** requests per minute | **2,000** requests per minute | +| **Authenticated** non-API HTTP traffic (for a given **user**) | **1,000** requests per minute | **1,000** requests per minute | +| **All** traffic (from a given **IP address**) | **2,000** requests per minute | **2,000** requests per minute | +| **Issue creation** | **300** requests per minute | **200** requests per minute | +| **Note creation** (on issues and merge requests) | **60** requests per minute | **60** requests per minute | +| **Advanced, project, and group search** API (for a given **IP address**) | **10** requests per minute | **10** requests per minute | +| **GitLab Pages** requests (for a given **IP address**) | | **1000** requests per **50 seconds** | +| **GitLab Pages** requests (for a given **GitLab Pages domain**) | | **5000** requests per **10 seconds** | +| **GitLab Pages** TLS connections (for a given **IP address**) | | **1000** requests per **50 seconds** | +| **GitLab Pages** TLS connections (for a given **GitLab Pages domain**) | | **400** requests per **10 seconds** | +| **Pipeline creation** requests (for a given **project, user, and commit**) | | **25** requests per minute | +| **Alert integration endpoint** requests (for a given **project**) | | **3600** requests per hour | More details are available on the rate limits for [protected paths](#protected-paths-throttle) and @@ -497,7 +464,8 @@ and can't be configured on GitLab.com to expire. You can erase job logs In addition to the GitLab Enterprise Edition Omnibus install, GitLab.com uses the following applications and settings to achieve scale. All settings are -publicly available at [chef cookbooks](https://gitlab.com/gitlab-cookbooks). +publicly available, as [Kubernetes configuration](https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com) +or [Chef cookbooks](https://gitlab.com/gitlab-cookbooks). ### Elastic cluster @@ -541,3 +509,10 @@ Service discovery: High Performance TCP/HTTP Load Balancer: - [`gitlab-cookbooks` / `gitlab-haproxy` · GitLab](https://gitlab.com/gitlab-cookbooks/gitlab-haproxy) + +## Sidekiq + +GitLab.com runs [Sidekiq](https://sidekiq.org) as an [external process](../../administration/sidekiq/index.md) +for Ruby job scheduling. + +The current settings are in the [GitLab.com Kubernetes pod configuration](https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com/-/blob/master/releases/gitlab/values/gprd.yaml.gotmpl). diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index 4aecf016e56..50506d005b0 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -37,12 +37,8 @@ The group's new subgroups have push rules set for them based on either: ## Restrict Git access protocols -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365601) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `group_level_git_protocol_control`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to -[enable the feature flag](../../administration/feature_flags.md) named `group_level_git_protocol_control`. On GitLab.com, -this feature is available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365601) in GitLab 15.1. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/365357) in GitLab 16.0. You can set the permitted protocols used to access a group's repositories to either SSH, HTTPS, or both. This setting is disabled when the [instance setting](../admin_area/settings/visibility_and_access_controls.md#configure-enabled-git-access-protocols) is @@ -64,7 +60,7 @@ To change the permitted Git access protocols for a group: To ensure only people from your organization can access particular resources, you can restrict access to groups by IP address. This group-level setting applies to: -- The GitLab UI, including subgroups, projects, and issues. +- The GitLab UI, including subgroups, projects, and issues. It does not apply to GitLab Pages. - [In GitLab 12.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/12874), the API. - In self-managed installations of GitLab 15.1 and later, you can also configure [globally-allowed IP address ranges](../admin_area/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges) @@ -90,8 +86,8 @@ To restrict group access by IP address: Keep in mind that restricting group access by IP address has the following implications: -- Administrators and group owners can access group settings from any IP address, regardless of IP restriction. However: - - Group owners can access the subgroups, but not the projects belonging to the group or subgroups, when accessing from a disallowed IP address. +- Administrators and group Owners can access group settings from any IP address, regardless of IP restriction. However: + - Group Owners can access the subgroups, but not the projects belonging to the group or subgroups, when accessing from a disallowed IP address. - Administrators can access projects belonging to the group when accessing from a disallowed IP address. Access to projects includes cloning code from them. - Users can still see group and project names and hierarchies. Only the following are restricted: @@ -102,8 +98,9 @@ Keep in mind that restricting group access by IP address has the following impli restricted IP address, the IP restriction prevents code from being cloned. - Users might still see some events from the IP-restricted groups and projects on their dashboard. Activity might include push, merge, issue, or comment events. -- IP access restrictions for Git operations via SSH are supported only on GitLab SaaS. - IP access restrictions applied to self-managed instances block SSH completely. +- IP access restrictions for Git operations via SSH are supported on GitLab SaaS. + IP access restrictions applied to self-managed instances are possible with [`gitlab-sshd`](../../administration/operations/gitlab_sshd.md) + with [PROXY protocol](../../administration/operations/gitlab_sshd.md#proxy-protocol-support) enabled. ## Restrict group access by domain **(PREMIUM)** @@ -181,12 +178,12 @@ prevent a project from being shared with other groups: 1. Select **Projects in `<group_name>` cannot be shared with other groups**. 1. Select **Save changes**. -This setting applies to all subgroups unless overridden by a group owner. Groups already +This setting applies to all subgroups unless overridden by a group Owner. Groups already added to a project lose access when the setting is enabled. ## Prevent users from requesting access to a group -As a group owner, you can prevent non-members from requesting access to +As a group Owner, you can prevent non-members from requesting access to your group. 1. On the top bar, **Main menu > Groups** and find your group. @@ -202,7 +199,7 @@ your group. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216987) in GitLab 13.3. By default, projects in a group can be forked. -Optionally, on [GitLab Premium](https://about.gitlab.com/pricing/) or higher tiers, +Optionally, on [GitLab Premium and Ultimate tiers](https://about.gitlab.com/pricing/), you can prevent the projects in a group from being forked outside of the current top-level group. This setting is removed from the SAML setting page, and migrated to the @@ -221,13 +218,13 @@ Existing forks are not removed. ## Prevent members from being added to projects in a group **(PREMIUM)** -As a group owner, you can prevent any new project membership for all +As a group Owner, you can prevent any new project membership for all projects in a group, allowing tighter control over project membership. For example, if you want to lock the group for an [Audit Event](../../administration/audit_events.md), you can guarantee that project membership cannot be modified during the audit. -If group membership lock is enabled, the group owner can still: +If group membership lock is enabled, the group Owner can still: - Invite groups or add members to groups to give them access to projects in the **locked** group. - Change the role of group members. @@ -286,7 +283,15 @@ To create group links via filter: LDAP user permissions can be manually overridden by an administrator. To override a user's permissions: 1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Group information > Members**. +1. On the left sidebar, select **Group information > Members**. If LDAP synchronization + has granted a user a role with: + - More permissions than the parent group membership, that user is displayed as having + [direct membership](../project/members/index.md#display-direct-members) of the group. + - The same or fewer permissions than the parent group membership, that user is displayed as having + [inherited membership](../project/members/index.md#display-inherited-members) of the group. +1. Optional. If the user you want to edit is displayed as having inherited membership, + [filter the subgroup to show direct members](manage.md#filter-a-group) before + overriding LDAP user permissions. 1. In the row for the user you are editing, select the pencil (**{pencil}**) icon. 1. Select **Edit permissions** in the modal. @@ -302,3 +307,17 @@ If a user sees a 404 when they would usually expect access, and the problem is l - `json.allowed`: `false` In viewing the log entries, compare `remote.ip` with the list of [allowed IP addresses](#restrict-group-access-by-ip-address) for the group. + +### Cannot update permissions for a group member + +If a group Owner cannot update permissions for a group member, check which memberships +are listed. Group Owners can only update direct memberships. + +If a parent group membership has the same or higher role than a subgroup, the +[inherited membership](../project/members/index.md#inherited-membership) is +listed on the subgroup members page, even if a [direct membership](../project/members/index.md#membership-types) +on the group exists. + +To view and update direct memberships, [filter the group to show direct members](manage.md#filter-a-group). + +The need to filter members by type through a redesigned members page that lists both direct and inherited memberships is proposed in [issue 337539](https://gitlab.com/gitlab-org/gitlab/-/issues/337539#note_1277786161). diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index cb760217487..4c448d8e5c2 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -1,11 +1,11 @@ --- type: reference -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Group-level Kubernetes clusters (certificate-based) (DEPRECATED) **(FREE)** +# Group-level Kubernetes clusters (certificate-based) (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34758) in GitLab 11.6. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/group/compliance_frameworks.md b/doc/user/group/compliance_frameworks.md index 84cca5800c2..2fca8b7b678 100644 --- a/doc/user/group/compliance_frameworks.md +++ b/doc/user/group/compliance_frameworks.md @@ -94,7 +94,8 @@ mutation { > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/331231) in GitLab 14.2. Group owners can configure a compliance pipeline in a project separate to other projects. By default, the compliance -pipeline configuration (`.gitlab-ci.yml` file) is run instead of the pipeline configuration of labeled projects. +pipeline configuration (for example, `.compliance-gitlab-ci.yml`) is run instead of the pipeline configuration (for example, `.gitlab-ci.yml`) of labeled +projects. However, the compliance pipeline configuration can reference the `.gitlab-ci.yml` file of the labeled projects so that: @@ -103,8 +104,11 @@ However, the compliance pipeline configuration can reference the `.gitlab-ci.yml - Jobs and variables defined in the compliance pipeline can't be changed by variables in the labeled project's `.gitlab-ci.yml` file. -See [example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from -labeled project pipeline configuration. +For more information, see: + +- [Example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from + labeled project pipeline configuration. +- The [Create a compliance pipeline](../../tutorials/compliance_pipeline/index.md) tutorial. ### Effect on labeled projects @@ -208,16 +212,47 @@ audit trail: - "# No after scripts." include: # Execute individual project's configuration (if project contains .gitlab-ci.yml) - project: '$CI_PROJECT_PATH' - file: '$CI_CONFIG_PATH' - ref: '$CI_COMMIT_SHA' # Must be defined or MR pipelines always use the use default branch - rules: - - if: $CI_PROJECT_PATH != "my-group/project-1" # Must be the hardcoded path to the project that hosts this configuration. + - project: '$CI_PROJECT_PATH' + file: '$CI_CONFIG_PATH' + ref: '$CI_COMMIT_SHA' # Must be defined or MR pipelines always use the use default branch + rules: + - if: $CI_PROJECT_PATH != "my-group/project-1" # Must be the hardcoded path to the project that hosts this configuration. ``` The `rules` configuration in the `include` definition avoids circular inclusion in case the compliance pipeline must be able to run in the host project itself. You can leave it out if your compliance pipeline only ever runs in labeled projects. +#### Compliance pipelines and custom pipeline configuration hosted externally + +The example above assumes that all projects host their pipeline configuration in the same project. +If any projects use [configuration hosted externally to the project](../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file): + +- The `include` section in the example compliance pipeline configuration must be adjusted. + For example, using [`include:rules`](../../ci/yaml/includes.md#use-rules-with-include): + + ```yaml + include: + # If the custom path variables are defined, include the project's external config file. + - project: '$PROTECTED_PIPELINE_CI_PROJECT_PATH' + file: '$PROTECTED_PIPELINE_CI_CONFIG_PATH' + ref: '$PROTECTED_PIPELINE_CI_REF' + rules: + - if: $PROTECTED_PIPELINE_CI_PROJECT_PATH && $PROTECTED_PIPELINE_CI_CONFIG_PATH && $PROTECTED_PIPELINE_CI_REF + # If any custom path variable is not defined, include the project's internal config file as normal. + - project: '$CI_PROJECT_PATH' + file: '$CI_CONFIG_PATH' + ref: '$CI_COMMIT_SHA' + rules: + - if: $PROTECTED_PIPELINE_CI_PROJECT_PATH == null || $PROTECTED_PIPELINE_CI_CONFIG_PATH == null || $PROTECTED_PIPELINE_CI_REF == null + ``` + +- [CI/CD variables](../../ci/variables/index.md) must be added to projects with external + pipeline configuration. In this example: + + - `PROTECTED_PIPELINE_CI_PROJECT_PATH`: The path to the project hosting the configuration file, for example `group/subgroup/project`. + - `PROTECTED_PIPELINE_CI_CONFIG_PATH`: The path to the configuration file in the project, for example `path/to/.gitlab-ci.yml`. + - `PROTECTED_PIPELINE_CI_REF`: The ref to use when retrieving the configuration file, for example `main`. + #### Compliance pipelines in merge requests originating in project forks When a merge request originates in a fork, the branch to be merged usually only exists in the fork. @@ -312,3 +347,53 @@ mutation { } } ``` + +### Compliance jobs are overwritten by target repository + +If you use the `extends` statement in a compliance pipeline configuration, compliance jobs are overwritten by the target repository job. For example, +you could have the following `.compliance-gitlab-ci.yml` configuration: + +```yaml +"compliance job": + extends: + - .compliance_template + stage: build + +.compliance_template: + script: + - echo "take compliance action" +``` + +You could also have the following `.gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: test + script: + - echo "overwriting compliance action" +``` + +This configuration results in the target repository pipeline overwriting the compliance pipeline, and you get the following message: +`overwriting compliance action`. + +To avoid overwriting a compliance job, don't use the `extends` keyword in compliance pipeline configuration. For example, +you could have the following `.compliance-gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: build + script: + - echo "take compliance action" +``` + +You could also have the following `.gitlab-ci.yml` configuration: + +```yaml +"compliance job": + stage: test + script: + - echo "overwriting compliance action" +``` + +This configuration doesn't overwrite the compliance pipeline and you get the following message: +`take compliance action`. diff --git a/doc/user/group/contribution_analytics/img/group_stats_cal.png b/doc/user/group/contribution_analytics/img/group_stats_cal.png Binary files differdeleted file mode 100644 index 0238c7bf088..00000000000 --- a/doc/user/group/contribution_analytics/img/group_stats_cal.png +++ /dev/null diff --git a/doc/user/group/contribution_analytics/img/group_stats_table.png b/doc/user/group/contribution_analytics/img/group_stats_table.png Binary files differdeleted file mode 100644 index 1f58b9717d0..00000000000 --- a/doc/user/group/contribution_analytics/img/group_stats_table.png +++ /dev/null diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md index 2716db27037..95d7ddb056e 100644 --- a/doc/user/group/custom_project_templates.md +++ b/doc/user/group/custom_project_templates.md @@ -1,7 +1,6 @@ --- -type: reference -stage: Manage -group: Import +stage: Create +group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -42,7 +41,7 @@ Projects in nested subgroups are not included in the template list. - Public and internal projects can be selected by any authenticated user as a template for a new project, if all [project features](../project/settings/index.md#configure-project-visibility-features-and-permissions) - except for **GitLab Pages** and **Security & Compliance** are set to **Everyone With Access**. + except for **GitLab Pages** and **Security and Compliance** are set to **Everyone With Access**. - Private projects can be selected only by users who are members of the projects. ## Example structure diff --git a/doc/user/group/devops_adoption/index.md b/doc/user/group/devops_adoption/index.md index a81b61c50ce..7105318e5df 100644 --- a/doc/user/group/devops_adoption/index.md +++ b/doc/user/group/devops_adoption/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group DevOps Adoption **(ULTIMATE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321083) in GitLab 13.11 as a [Beta feature](../../../policy/alpha-beta-support.md#beta-features). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321083) in GitLab 13.11 as a [Beta feature](../../../policy/alpha-beta-support.md#beta). > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333556) in GitLab 14.1. > - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. > - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md index 64addd524ad..6783e89779b 100644 --- a/doc/user/group/epics/epic_boards.md +++ b/doc/user/group/epics/epic_boards.md @@ -8,17 +8,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5067) in GitLab 13.10. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290039) in GitLab 14.1. +> - Displaying total weight on the top of lists [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/364503) in GitLab 15.11. Epic boards build on the existing [epic tracking functionality](index.md) and [labels](../../project/labels.md). Your epics appear as cards in vertical lists, organized by their assigned labels. +On the top of each list, you can see the number of epics in the list (**{epic}**) and the total weight of all its epics (**{weight}**). + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=I1bFIAQBHB8">Epics and Issue Boards - Project Management</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/I1bFIAQBHB8" frameborder="0" allowfullscreen> </iframe> +</figure> + To view an epic board: 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics > Boards**. -![GitLab epic board - Premium](img/epic_board_v14_1.png) +![GitLab epic board - Premium](img/epic_board_v15_10.png) ## Create an epic board @@ -30,7 +40,7 @@ To create a new epic board: 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Epics > Boards**. -1. In the upper left corner, select the dropdown list with the current board name. +1. In the upper-left corner, select the dropdown list with the current board name. 1. Select **Create new board**. 1. Enter the new board's title. 1. Optional. To hide the Open or Closed lists, clear the **Show the Open list** and @@ -54,7 +64,7 @@ Prerequisites: To delete the active epic board: -1. Select the dropdown list with the current board name in the upper left corner of the epic boards page. +1. In the upper-left corner of the epic board page, select the dropdown list. 1. Select **Delete board**. 1. Select **Delete**. @@ -63,7 +73,7 @@ To delete the active epic board: - [Create a new list](#create-a-new-list). - [Remove an existing list](#remove-a-list). - [Filter epics](#filter-epics). -- Create workflows, like when using [issue boards](../../project/issue_board.md#create-workflows). +- Create workflows, like when using [issue boards](../../project/issue_board.md#issue-board-workflow-between-teams). - [Move epics and lists](#move-epics-and-lists). - Change epic labels (by dragging an epic between lists). - Close an epic (by dragging it to the **Closed** list). @@ -100,7 +110,7 @@ To remove a list from an epic board: 1. Select **Remove list**. A confirmation dialog appears. 1. Select **OK**. -## Create an epic from an epic board +### Create an epic from an epic board > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233568) in GitLab 14.0. @@ -115,7 +125,7 @@ To create an epic from a list in epic board: 1. Enter the new epic's title. 1. Select **Create epic**. -![Create a GitLab epic from an epic board](img/epic_board_epic_create_v14_1.png) +![Create a GitLab epic from an epic board](img/epic_board_epic_create_v15_10.png) ### Filter epics diff --git a/doc/user/group/epics/img/button_close_epic.png b/doc/user/group/epics/img/button_close_epic.png Binary files differdeleted file mode 100644 index aa1a889ea23..00000000000 --- a/doc/user/group/epics/img/button_close_epic.png +++ /dev/null diff --git a/doc/user/group/epics/img/epic_board_epic_create_v14_1.png b/doc/user/group/epics/img/epic_board_epic_create_v14_1.png Binary files differdeleted file mode 100644 index 04017014885..00000000000 --- a/doc/user/group/epics/img/epic_board_epic_create_v14_1.png +++ /dev/null diff --git a/doc/user/group/epics/img/epic_board_epic_create_v15_10.png b/doc/user/group/epics/img/epic_board_epic_create_v15_10.png Binary files differnew file mode 100644 index 00000000000..71d1fc0a9fc --- /dev/null +++ b/doc/user/group/epics/img/epic_board_epic_create_v15_10.png diff --git a/doc/user/group/epics/img/epic_board_v14_1.png b/doc/user/group/epics/img/epic_board_v14_1.png Binary files differdeleted file mode 100644 index ccf1ef9559e..00000000000 --- a/doc/user/group/epics/img/epic_board_v14_1.png +++ /dev/null diff --git a/doc/user/group/epics/img/epic_board_v15_10.png b/doc/user/group/epics/img/epic_board_v15_10.png Binary files differnew file mode 100644 index 00000000000..03bc96d9623 --- /dev/null +++ b/doc/user/group/epics/img/epic_board_v15_10.png diff --git a/doc/user/group/epics/img/issue_list_v13_1.png b/doc/user/group/epics/img/issue_list_v13_1.png Binary files differdeleted file mode 100644 index ed0b4842bfe..00000000000 --- a/doc/user/group/epics/img/issue_list_v13_1.png +++ /dev/null diff --git a/doc/user/group/epics/img/issue_list_v15_11.png b/doc/user/group/epics/img/issue_list_v15_11.png Binary files differnew file mode 100644 index 00000000000..601406a8a33 --- /dev/null +++ b/doc/user/group/epics/img/issue_list_v15_11.png diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 21c95f37aeb..32454693d71 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -59,7 +59,7 @@ have a start or due date, a visual - Link [related epics](linked_epics.md) based on a type of relationship. - Create workflows with [epic boards](epic_boards.md). - [Turn on notifications](../../profile/notifications.md) for about epic events. -- [Award an emoji](../../award_emojis.md) to an epic or its comments. +- [Add an emoji reaction](../../award_emojis.md) to an epic or its comments. - Collaborate on an epic by posting comments in a [thread](../../discussions/index.md). - Use [health status](../../project/issues/managing_issues.md#health-status) to track your progress. diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 74cfa2bd6ed..98316188496 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -41,8 +41,6 @@ The newly created epic opens. ### Start and due date inheritance -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7332) in GitLab 12.5 to replace **From milestones**. - If you select **Inherited**: - For the **start date**: GitLab scans all child epics and issues assigned to the epic, @@ -52,7 +50,7 @@ If you select **Inherited**: and sets the due date to match the latest due date found in the child epics or the milestone assigned to the issues. -These are dynamic dates and recalculated if any of the following occur: +These dates are dynamic and recalculated if any of the following occur: - A child epic's dates change. - Milestones are reassigned to an issue. @@ -123,8 +121,6 @@ To reorder list items, when viewing an epic: ## Bulk edit epics -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7250) in GitLab 12.2. - Users with at least the Reporter role can manage epics. When bulk editing epics in a group, you can edit their labels. @@ -136,7 +132,7 @@ Prerequisites: To update multiple epics at the same time: 1. In a group, go to **Epics > List**. -1. Select **Edit epics**. A sidebar on the right appears with editable fields. +1. Select **Bulk edit**. A sidebar on the right appears with editable fields. 1. Select the checkboxes next to each epic you want to edit. 1. Select the appropriate fields and their values from the sidebar. 1. Select **Update all**. @@ -163,14 +159,13 @@ Prerequisites: - You must have at least the Reporter role for the epic's group. -Whenever you decide that there is no longer need for that epic, -close the epic by: - -- Selecting **Close epic**. +To close an epic, at the top of an epic, select **Close epic**. - ![close epic - button](img/button_close_epic.png) +<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> +If you don't see this action at the top of an epic, your project or instance might have +enabled a feature flag for [moved actions](../../project/merge_requests/index.md#move-sidebar-actions) -- Using the `/close` [quick action](../../project/quick_actions.md). +You can also use the `/close` [quick action](../../project/quick_actions.md). ## Reopen a closed epic @@ -233,7 +228,6 @@ than 1000. The cached value is rounded to thousands or millions and updated ever ## Filter the list of epics -> - Filtering by epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in GitLab 12.9. > - Filtering by child epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in GitLab 13.0. > - Filtering by the user's reaction emoji [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325630) in GitLab 13.11. > - Sorting by epic titles [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.1. @@ -341,7 +335,7 @@ in relation to epics. ### View issues assigned to an epic -On the **Epics and Issues** tab, you can see epics and issues assigned to this epic. +On the **Child issues and epics** section, you can see epics and issues assigned to this epic. Only epics and issues that you can access show on the list. You can always view the issues assigned to the epic if they are in the group's child project. @@ -350,7 +344,7 @@ of its parent group. ### View count of issues in an epic -On the **Epics and Issues** tab, under each epic name, hover over the total counts. +On the **Child issues and epics** section, under each epic name, hover over the total counts. The number indicates all epics associated with the project, including issues you might not have permission to. @@ -365,7 +359,7 @@ automatically added to the epic. > Minimum required role for the project [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382506) from Reporter to Guest in GitLab 15.8. You can add existing issues to an epic, including issues in a project from a [different group hierarchy](index.md#child-issues-from-different-group-hierarchies). -Newly added issues appear at the top of the list of issues in the **Epics and Issues** tab. +Newly added issues appear at the top of the list of issues in the **Child issues and epics** section. An epic contains a list of issues and an issue can be associated with at most one epic. When you add a new issue that's already linked to an epic, the issue is automatically unlinked from its @@ -377,13 +371,13 @@ Prerequisites: To add an existing issue to an epic: -1. On the epic's page, under **Epics and Issues**, select **Add**. +1. On the epic's page, under **Child issues and epics**, select **Add**. 1. Select **Add an existing issue**. 1. Identify the issue to be added, using either of the following methods: - Paste the link of the issue. - Search for the desired issue by entering part of the issue's title, then selecting the desired - match ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9126) in GitLab 12.5). Issues - from different group hierarchies do not appear in search results. To add such an issue, enter its full URL. + match. Issues from different group hierarchies do not appear in search results. + To add such an issue, enter its full URL. If there are multiple issues to be added, press <kbd>Space</kbd> and repeat this step. 1. Select **Add**. @@ -401,7 +395,7 @@ Prerequisites: To create an issue from an epic: -1. On the epic's page, under **Epics and Issues**, select **Add**. +1. On the epic's page, under **Child issues and epics**, select **Add**. 1. Select **Add a new issue**. 1. Under **Title**, enter the title for the new issue. 1. From the **Project** dropdown list, select the project in which the issue should be created. @@ -426,14 +420,13 @@ To remove an issue from an epic: The **Remove issue** warning appears. 1. Select **Remove**. -![List of issues assigned to an epic](img/issue_list_v13_1.png) +![List of issues assigned to an epic](img/issue_list_v15_11.png) ### Reorder issues assigned to an epic -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9367) in GitLab 12.5. -> - Minimum required role for the project [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382506) from Reporter to Guest in GitLab 15.8. +> Minimum required role for the project [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382506) from Reporter to Guest in GitLab 15.8. -New issues appear at the top of the list in the **Epics and Issues** tab. +New issues appear at the top of the list in the **Child issues and epics** section. You can reorder the list of issues by dragging them. Prerequisites: @@ -442,7 +435,7 @@ Prerequisites: To reorder issues assigned to an epic: -1. Go to the **Epics and Issues** tab. +1. Go to the **Child issues and epics** section. 1. Drag issues into the desired order. ### Move issues between epics **(ULTIMATE)** @@ -450,7 +443,7 @@ To reorder issues assigned to an epic: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33039) in GitLab 13.0. > - Minimum required role for the project [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382506) from Reporter to Guest in GitLab 15.8. -New issues appear at the top of the list in the **Epics and Issues** +New issues appear at the top of the list in the **Child issues and epics** tab. You can move issues from one epic to another. Prerequisites: @@ -459,13 +452,12 @@ Prerequisites: To move an issue to another epic: -1. Go to the **Epics and Issues** tab. +1. Go to the **Child issues and epics** section. 1. Drag issues into the desired parent epic in the visible hierarchy. ### Promote an issue to an epic -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3777) in GitLab 11.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) from GitLab Ultimate to GitLab Premium in 12.8. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) from GitLab Ultimate to GitLab Premium in 12.8. Prerequisites: @@ -504,12 +496,12 @@ You can create a spreadsheet template to manage a pattern of consistently repeat <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an introduction to epic templates, see [GitLab Epics and Epic Template Tip](https://www.youtube.com/watch?v=D74xKFNw8vg). -For more on epic templates, see [Epic Templates - Repeatable sets of issues](https://about.gitlab.com/handbook/marketing/strategic-marketing/getting-started/104/). +For more on epic templates, see [Epic Templates - Repeatable sets of issues](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/getting-started/104/). ## Multi-level child epics **(ULTIMATE)** You can add any epic that belongs to a group or subgroup of the parent epic's group. -New child epics appear at the top of the list of epics in the **Epics and Issues** tab. +New child epics appear at the top of the list of epics in the **Child issues and epics** section. When you add an epic that's already linked to a parent epic, the link to its current parent is removed. @@ -522,11 +514,7 @@ The maximum number of direct child epics is 100. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8502) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `child_epics_from_different_hierarchies`. Disabled by default. > - Minimum required role for the group [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382503) from Reporter to Guest in GitLab 15.7. > - Cross-group child epics [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/375622) in GitLab 15.9. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is available. To disable it, -ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `child_epics_from_different_hierarchies`. -On GitLab.com, this feature is available. +> - [Feature flag `child_epics_from_different_hierarchies`](https://gitlab.com/gitlab-org/gitlab/-/issues/382719) removed in GitLab 15.10. You can add a child epic that belongs to a group that is different from the parent epic's group. @@ -548,7 +536,7 @@ Prerequisites: To add a new epic as child epic: 1. In an epic, in the **Child issues and epics** section, select **Add > Add a new epic**. -1. Select a group from the dropdown. The epic's group is selected by default. +1. Select a group from the dropdown list. The epic's group is selected by default. 1. Enter a title for the new epic. 1. Select **Create epic**. @@ -557,7 +545,7 @@ To add an existing epic as child epic: 1. In an epic, in the **Child issues and epics** section, select **Add > Add an existing epic**. 1. Identify the epic to be added, using either of the following methods: - Paste the link of the epic. - - Search for the desired issue by entering part of the epic's title, then selecting the desired match. This search is only available for epics within the same group hierarchy. + - Search for the desired issue by entering part of the epic's title, then selecting the desired match. This search is only available for epics in the same group hierarchy. If there are multiple epics to be added, press <kbd>Space</kbd> and repeat this step. 1. Select **Add**. @@ -567,7 +555,7 @@ To add an existing epic as child epic: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33039) in GitLab 13.0. > - Minimum required role for the group [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382503) from Reporter to Guest in GitLab 15.7. -New child epics appear at the top of the list in the **Epics and Issues** tab. +New child epics appear at the top of the list in the **Child issues and epics** section. You can move child epics from one epic to another. When you add a new epic that's already linked to a parent epic, the link to its current parent is removed. Issues and child epics cannot be intermingled. @@ -578,15 +566,14 @@ Prerequisites: To move child epics to another epic: -1. Go to the **Epics and Issues** tab. +1. Go to the **Child issues and epics** section. 1. Drag epics into the desired parent epic. ### Reorder child epics assigned to an epic -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9367) in GitLab 12.5. -> - Minimum required role for the group [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382503) from Reporter to Guest in GitLab 15.7. +> Minimum required role for the group [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/382503) from Reporter to Guest in GitLab 15.7. -New child epics appear at the top of the list in the **Epics and Issues** tab. +New child epics appear at the top of the list in the **Child issues and epics** section. You can reorder the list of child epics. Prerequisites: @@ -595,7 +582,7 @@ Prerequisites: To reorder child epics assigned to an epic: -1. Go to the **Epics and Issues** tab. +1. Go to the **Child issues and epics** section. 1. Drag epics into the desired order. ### Remove a child epic from a parent epic diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index eb32856ff79..7f55bf56102 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -27,42 +27,55 @@ If you migrate from GitLab.com to self-managed GitLab, an administrator can crea > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) in GitLab 14.4 for project resources [with a flag](../../feature_flags.md) named `bulk_import_projects`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.6. > - New application setting `bulk_import_enabled` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383268) in GitLab 15.8. `bulk_import` feature flag removed. +> - `bulk_import_projects` feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.10. -FLAG: On self-managed GitLab, by default [migrating group items](#migrated-group-items) is not available. To show the feature, ask an administrator to [enable it in application settings](../../admin_area/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer). -Also on self-managed GitLab, by default [migrating project items](#migrated-project-items-beta) is not available. To show -this feature, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`bulk_import_projects`. The feature is not ready for production use. On GitLab.com, migration of both groups and projects is available. Migrating groups by direct transfer copies the groups from one place to another. You can: - Copy many groups at once. -- Copy top-level groups to: +- In the GitLab UI, copy top-level groups to: - Another top-level group. - The subgroup of any existing top-level group. - Another GitLab instance, including GitLab.com. -- Copy groups with projects (in [beta](../../../policy/alpha-beta-support.md#beta-features) and not ready for production +- In the [API](../../../api/bulk_imports.md), copy top-level groups and subgroups to these locations. +- Copy groups with projects (in [Beta](../../../policy/alpha-beta-support.md#beta) and not ready for production use) or without projects. Copying projects with groups is available: - On GitLab.com by default. - - On self-managed GitLab instances after an administrator first [enables the feature flag](../../../administration/feature_flags.md) named `bulk_import_projects`. Not all group and project resources are copied. See list of copied resources below: - [Migrated group items](#migrated-group-items). - [Migrated project items](#migrated-project-items-beta). +WARNING: +Importing groups with projects is in [Beta](../../../policy/alpha-beta-support.md#beta). This feature is not +ready for production use. + We invite you to leave your feedback about migrating by direct transfer in [the feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/284495). If you want to move groups instead of copying groups, you can [transfer groups](../manage.md#transfer-a-group) if the groups are in the same GitLab instance. Transferring groups is a faster and more complete option. -### Rate limit +### Known issues + +See [epic 6629](https://gitlab.com/groups/gitlab-org/-/epics/6629) for a list of known issues for migrating by direct +transfer. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386452) in GitLab 15.9. +### Limits -Each user can perform up to six migrations per minute. +| Limit | Description | +|:------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 6 | Maximum number of migrations permitted by a destination GitLab instance per minute per user. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386452) in GitLab 15.9. | +| 5 GB | Maximum relation size that can be downloaded from the source instance. | +| 10 GB | Maximum size of a decompressed archive. | +| 210 seconds | Maximum number of seconds to wait for decompressing an archive file. | +| 50 MB | Maximum length an NDJSON row can have. | +| 5 minutes | Maximum number of seconds until an empty export status on source instance is raised. | +| 8 hours | Time until migration times out. | +| 90 minutes | Time the destination is waiting for export to complete. | ### Visibility rules @@ -78,6 +91,8 @@ make sure to have a similar setup on the destination instance, or to import into ### Prerequisites +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + To migrate groups by direct transfer: - The network connection between instances or GitLab.com must support HTTPS. @@ -92,25 +107,22 @@ To migrate groups by direct transfer: - For GitLab 15.0 and earlier source instances, the personal access token must have both the `api` and `read_repository` scopes. - You must have the Owner role on the source group to migrate from. -- You must have at least the Maintainer role on the destination group to migrate to. Using the Developer role for this - purpose was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in - GitLab 16.0. +- You must have at least the Maintainer role on the destination group to migrate to. ### Prepare user accounts To ensure GitLab maps users and their contributions correctly: -1. Create the required users on the destination GitLab instance. When migrating to GitLab.com, you must create users - manually unless [SCIM](../../group/saml_sso/scim_setup.md) is used. Creating users with the API is only available to - self-managed instances because it requires administrator access. -1. Check that users have a public email on the source GitLab instance that matches their primary email on the - destination GitLab instance. -1. Ensure that users confirm their primary email addresses on the destination GitLab instance. Most users receive an - email asking them to confirm their email address. -1. If using an OmniAuth provider like SAML, link GitLab and SAML accounts of users on GitLab. All users on the - destination GitLab instance must sign in and verify their account on the destination GitLab instance. If using - [SAML SSO for GitLab.com groups](../../group/saml_sso/index.md), users must - [link their SAML identity to their GitLab.com account](../../group/saml_sso/index.md#linking-saml-to-your-existing-gitlabcom-account). +1. Create the required users on the destination GitLab instance. You can create users with the API only on self-managed instances because it requires + administrator access. When migrating to GitLab.com or a self-managed GitLab instance you can: + - Create users manually. + - Set up or use your existing [SAML SSO provider](../saml_sso/index.md) and leverage user synchronization of SAML SSO groups supported through + [SCIM](../../group/saml_sso/scim_setup.md). You can + [bypass the GitLab user account verification with verified email domains](../saml_sso/index.md#bypass-user-email-confirmation-with-verified-domains). +1. Ensure that users have a [public email](../../profile/index.md#set-your-public-email) on the source GitLab instance that matches any confirmed email address on the destination GitLab instance. Most + users receive an email asking them to confirm their email address. +1. If users already exist on the destination instance and you use [SAML SSO for GitLab.com groups](../../group/saml_sso/index.md), all users must + [link their SAML identity to their GitLab.com account](../../group/saml_sso/index.md#link-saml-to-your-existing-gitlabcom-account). ### Connect the source GitLab instance @@ -125,7 +137,7 @@ Create the group you want to import to and connect the source GitLab instance: 1. Enter the [personal access token](../../../user/profile/personal_access_tokens.md) for your source GitLab instance. 1. Select **Connect instance**. -### Select the groups to import +### Select the groups and projects to import > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385689) in GitLab 15.8, option to import groups with or without projects. @@ -135,12 +147,15 @@ role. 1. By default, the proposed group namespaces match the names as they exist in source instance, but based on your permissions, you can choose to edit these names before you proceed to import any of them. 1. Next to the groups you want to import, select either: - - **Import with projects**. Importing groups with projects is in [Beta](../../../policy/alpha-beta-support.md#beta-features). This feature is not ready for production use. + - **Import with projects**. - **Import without projects**. - - **Import** on self-managed GitLab, when the `bulk_import_projects` feature flag is disabled and the feature is not available. 1. The **Status** column shows the import status of each group. If you leave the page open, it updates in real-time. 1. After a group has been imported, select its GitLab path to open its GitLab URL. +WARNING: +Importing groups with projects is in [Beta](../../../policy/alpha-beta-support.md#beta). This feature is not +ready for production use. + ### Group import history You can view all groups migrated by you by direct transfer listed on the group import history page. This list includes: @@ -157,108 +172,162 @@ To view group import history: 1. On the top bar, select **Create new…** (**{plus-square}**). 1. Select **New group**. 1. Select **Import group**. -1. Select **History** in the upper right corner. +1. In the upper-right corner, select **History**. 1. If there are any errors for a particular import, you can see them by selecting **Details**. ### Migrated group items -The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/group/import_export.yml) -file for groups lists many of the items imported when migrating groups by direct transfer. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, -[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/group/import_export.yml). +The group items that are migrated depend on the version of GitLab you use on the destination. To determine if a +specific group item is migrated: + +1. Check the [`groups/stage.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/bulk_imports/groups/stage.rb) + file for all editions and the + [`groups/stage.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/bulk_imports/groups/stage.rb) file + for Enterprise Edition for your version on the destination. For example, for version 15.9: + - <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/bulk_imports/groups/stage.rb> (all editions). + - <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/ee/lib/ee/bulk_imports/groups/stage.rb> (Enterprise + Edition). +1. Check the + [`group/import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/group/import_export.yml) + file for groups for your version on the destination. For example, for version 15.9: + <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/gitlab/import_export/group/import_export.yml>. + +Any other group items are **not** migrated. Group items that are migrated to the destination GitLab instance include: -- Badges ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292431) in 13.11) -- Board Lists -- Boards -- Epics ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250281) in 13.7) - - Epic resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) -- Finisher -- Group Labels ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292429) in 13.9) -- Iterations ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292428) in 13.10) -- Iterations cadences ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96570) in 15.4) -- Members ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299415) in 13.9) - Group members are associated with the imported group if: - - The user already exists in the destination GitLab instance and - - The user has a public email in the source GitLab instance that matches a - confirmed email in the destination GitLab instance -- Milestones ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292427) in 13.10) -- Namespace Settings -- Releases - - Milestones ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) in GitLab 15.0). -- Subgroups -- Uploads - -Any other items are **not** migrated. - -### Migrated project items (beta) +| Group item | Introduced in | +|:---------------------|:----------------------------------------------------------------------------| +| Badges | [GitLab 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/292431) | +| Boards | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18938) | +| Board lists | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24863) | +| Epics <sup>1</sup> | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/issues/250281) | +| Group labels | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/292429) | +| Iterations | [GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/292428) | +| Iteration cadences | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96570) | +| Members <sup>2</sup> | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/299415) | +| Group milestones | [GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/292427) | +| Namespace settings | [GitLab 14.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85128) | +| Release milestones | [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) | +| Subgroups | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18938) | +| Uploads | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18938) | + +1. Epic resource state events [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4, label + associations [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62074) in GitLab 13.12, state and + state ID [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28203) in GitLab 13.7, and system note + metadata [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63551) in GitLab 14.0. +1. Group members are associated with the imported group if the user: + - Already exists in the destination GitLab instance. + - Has a public email in the source GitLab instance that matches a confirmed email in the destination GitLab instance. + +### Migrated project items (Beta) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) in GitLab 14.4 [with a flag](../../feature_flags.md) named `bulk_import_projects`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.6. +> - `bulk_import_projects` feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.10. +> - Project-only migrations using API [added](https://gitlab.com/gitlab-org/gitlab/-/issues/390515) in GitLab 15.11. + +If you choose to migrate projects when you [select groups to migrate](#select-the-groups-and-projects-to-import), +project items are migrated with the projects. + +The project items that are migrated depends on the version of GitLab you use on the destination. To determine if a +specific project item is migrated: -FLAG: -On self-managed GitLab, migrating project resources when migrating groups is not available by default. -To make it available ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`bulk_import_projects`. On GitLab.com, groups are migrated with all their projects by default. +1. Check the [`projects/stage.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/bulk_imports/projects/stage.rb) + file for all editions and the + [`projects/stage.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/bulk_imports/projects/stage.rb) + file for Enterprise Edition for your version on the destination. For example, for version 15.9: + - <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/bulk_imports/projects/stage.rb> (all editions). + - <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/ee/lib/ee/bulk_imports/projects/stage.rb> (Enterprise + Edition). +1. Check the + [`project/import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) + file for projects for your version on the destination. For example, for version 15.9: + <https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/gitlab/import_export/project/import_export.yml>. -The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) -file for projects lists many of the items imported when migrating projects using group migration. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, -[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/project/import_export.yml). +Any other project items are **not** migrated. + +If you choose not to migrate projects along with groups or if you want to retry a project migration, you can +initiate project-only migrations using the [API](../../../api/bulk_imports.md). WARNING: -Migrating projects when migrating groups by direct transfer is in [Beta](../../../policy/alpha-beta-support.md#beta-features) +Migrating projects when migrating groups by direct transfer is in [Beta](../../../policy/alpha-beta-support.md#beta) and is not ready for production use. Project items that are migrated to the destination GitLab instance include: -- Projects ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) in GitLab 14.4) - - Auto DevOps ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339410) in GitLab 14.6) - - Badges ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75029) in GitLab 14.6) - - Branches (including protected branches) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339414) in GitLab 14.7) - - CI Pipelines ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339407) in GitLab 14.6) - - Designs ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339421) in GitLab 15.1) - - Issues ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) in GitLab 14.4) - - Issue iteration ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in 15.4) - - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource iteration events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Merge request URL references ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) in GitLab 15.6) - - Time Tracking ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) in GitLab 14.4) - - Issue boards ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71661) in GitLab 14.4) - - Labels ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339419) in GitLab 14.4) - - LFS Objects ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339405) in GitLab 14.8) - - Members ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341886) in GitLab 14.8) - - Merge Requests ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.5) - - Multiple merge request assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Merge request resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue URL references ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) in GitLab 15.6) - - Time Tracking ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.5) - - Push Rules ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.6) - - Milestones ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339417) in GitLab 14.5) - - External Pull Requests ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339409) in GitLab 14.5) - - Pipeline History ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339412) in GitLab 14.6) - - Pipeline Schedules ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339408) in GitLab 14.8) - - Project Features ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74722) in GitLab 14.6) - - Releases ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) in GitLab 15.1) - - Release Evidences ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360567) in GitLab 15.1) - - Repositories ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) in GitLab 14.4) - - Snippets ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343438) in GitLab 14.6) - - Settings ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339416) in GitLab 14.6) - - Avatar ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75249) in GitLab 14.6) - - Container Expiration Policy ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75653) in GitLab 14.6) - - Project Properties ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75898) in GitLab 14.6) - - Service Desk ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75653) in GitLab 14.6) - - Uploads ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339401) in GitLab 14.5) - - Wikis ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345923) in GitLab 14.6) - -Items excluded from migration, because they contain sensitive information: - -- Pipeline Triggers. +| Project item | Introduced in | +|:----------------------------------------|:---------------------------------------------------------------------------| +| Projects | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) | +| Auto DevOps | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339410) | +| Badges | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75029) | +| Branches (including protected branches) | [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/339414) | +| CI Pipelines | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339407) | +| Commit comments | [GitLab 15.10](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) | +| Designs | [GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/339421) | +| Issues | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) | +| Issue boards | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71661) | +| Labels | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/339419) | +| LFS Objects | [GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/339405) | +| Members | [GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/341886) | +| Merge requests | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) | +| Push rules | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) | +| Milestones | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339417) | +| External pull requests | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339409) | +| Pipeline history | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339412) | +| Pipeline schedules | [GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/339408) | +| Project features | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74722) | +| Releases | [GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) | +| Release evidences | [GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/360567) | +| Repositories | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267945) | +| Snippets | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/343438) | +| Settings | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/339416) | +| Uploads | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339401) | +| Wikis | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/issues/345923) | + +#### Issue-related items + +Issue-related project items that are migrated to the destination GitLab instance include: + +| Issue-related project item | Introduced in | +|:--------------------------------|:---------------------------------------------------------------------------| +| Issue iterations | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) | +| Issue resource state events | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) | +| Issue resource milestone events | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) | +| Issue resource iteration events | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) | +| Merge request URL references | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) | +| Time tracking | [GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/267946) | + +#### Merge request-related items + +Merge request-related project items that are migrated to the destination GitLab instance include: + +| Merge request-related project item | Introduced in | +|:----------------------------------------|:--------------------------------------------------------------------| +| Multiple merge request assignees | [GitLab 15.3](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) | +| Merge request reviewers | [GitLab 15.3](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) | +| Merge request approvers | [GitLab 15.3](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) | +| Merge request resource state events | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) | +| Merge request resource milestone events | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) | +| Issue URL references | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) | +| Time tracking | [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) | + +#### Setting-related items + +Setting-related project items that are migrated to the destination GitLab instance include: + +| Setting-related project item | Introduced in | +|:-----------------------------|:---------------------------------------------------------------------------| +| Avatar | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75249) | +| Container expiration policy | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75653) | +| Project properties | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75898) | +| Service Desk | [GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75653) | + +#### Excluded items + +Project items excluded from migration because they contain sensitive information: + +- Pipeline triggers. ### Troubleshooting @@ -283,7 +352,7 @@ entities.where(status: [-1]).pluck(:destination_name, :destination_namespace, :s ``` You can also see all migrated entities with any failures related to them using an -[API endpoint](../../../api/bulk_imports.md#list-all-group-migrations-entities). +[API endpoint](../../../api/bulk_imports.md#list-all-group-or-project-migrations-entities). #### Stale imports @@ -316,6 +385,18 @@ To solve this, you must change the source group path to include a non-numerical - The [Groups API](../../../api/groups.md#update-group). +#### Other `404` errors + +You can receive other `404` errors when importing a group, for example: + +```json +"exception_message": "Unsuccessful response 404 from [FILTERED] Bo...", +"exception_class": "BulkImports::NetworkError", +``` + +This error indicates a problem transferring from the _source_ instance. To solve this, check that you have met the [prerequisites](#prerequisites) on the source +instance. + ## Migrate groups by uploading an export file (deprecated) > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2888) in GitLab 13.0 as an experimental feature. May change in future releases. @@ -323,8 +404,8 @@ To solve this, you must change the source group path to include a non-numerical WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/4619) in GitLab 14.6 and replaced by -[migrating groups by direct transfer](#migrate-groups-by-direct-transfer-recommended). To follow progress on a solution for -[offline environments](../../application_security/offline_deployments/index.md), see +[migrating groups by direct transfer](#migrate-groups-by-direct-transfer-recommended). However, this feature is still recommended for migrating groups between +offline systems. To follow progress on an alternative solution for [offline environments](../../application_security/offline_deployments/index.md), see [the relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/8985). Prerequisites: @@ -356,13 +437,11 @@ Note the following: ### Compatibility -Group file exports are in NDJSON format. GitLab previously produced group file exports in JSON format, however: +> Support for JSON-formatted project file exports [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/383682) in GitLab 15.8. -- From GitLab 15.8, GitLab no longer supports importing a JSON-formatted group file export. -- Between GitLab 14.0 and GitLab 14.7, GitLab no longer produces group file exports in JSON format but, to support - transitions, can still import JSON-formatted group file exports. +Group file exports are in NDJSON format. -From GitLab 13.0, GitLab can import group file exports that were exported from a version of GitLab up to two +You can import group file exports that were exported from a version of GitLab up to two [minor](../../../policy/maintenance.md#versioning) versions behind, which is similar to our process for [security releases](../../../policy/maintenance.md#security-releases). @@ -377,7 +456,7 @@ For example: The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/group/import_export.yml) file for groups lists items exported and imported when migrating groups using file exports. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, +for your version of GitLab to check which items can be imported to the destination GitLab instance. For example, [`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/group/import_export.yml). Group items that are exported include: @@ -404,7 +483,7 @@ Items that are **not** exported include: - To preserve the member list and their respective permissions on imported groups, review the users in these groups. Make sure these users exist before importing the desired groups. -- Users must set a public email in the source GitLab instance that matches one of their verified emails in the target GitLab instance. +- Users must set a public email in the source GitLab instance that matches their confirmed primary email in the destination GitLab instance. Most users receive an email asking them to confirm their email address. ### Enable export for a group diff --git a/doc/user/group/index.md b/doc/user/group/index.md index db01358d899..7e093ccb01b 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -20,6 +20,16 @@ For larger organizations, you can also create [subgroups](subgroups/index.md). For more information about creating and managing your groups, see [Manage groups](manage.md). +NOTE: +Self-managed customers should create a top-level group so you can see an overview of +your organization. For more information about efforts to create an +organization view of all groups, [see epic 9266](https://gitlab.com/groups/gitlab-org/-/epics/9266). +A top-level group can also have one complete +[Security Dashboard and Center](../application_security/security_dashboard/index.md), +[Vulnerability](../application_security/vulnerability_report/index.md#vulnerability-report) and +[Compliance Report](../compliance/compliance_report/index.md), and +[Value stream analytics](../group/value_stream_analytics/index.md). + ## Group visibility Like projects, a group can be configured to limit the visibility of it to: diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index 9eb6d4387c1..17aa6cb9655 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -67,7 +67,7 @@ To configure group insights: in a project that belongs to your group. 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > General**. -1. Expand **Insights**. +1. Expand the **Analytics** tab and find the **Insights** section. 1. Select the project that contains your `.gitlab/insights.yml` configuration file. 1. Select **Save changes**. diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 72d3bf65447..9b246e6ad47 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -62,6 +62,8 @@ To create an iteration cadence: - From the **Upcoming iterations** dropdown list, select how many upcoming iterations should be created and maintained by GitLab. - Optional. To move incomplete issues to the next iteration, select **Roll over issues**. + At the end of the current iteration, all open issues are added to the next iteration. + Issues are moved at midnight in the instance time zone (UTC by default). Administrators can change the instance time zone. 1. Select **Create cadence**. The cadence list page opens. If you want to manually manage the created cadence, read [Manual Iteration Management](#manual-iteration-management). diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index d21dbe357da..7c2c2eaa211 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -8,6 +8,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use groups to manage one or more related projects at the same time. +NOTE: +Self-managed customers should create a top-level group so you can see an overview of +your organization. For more information about efforts to create an +organization view of all groups, [see epic 9266](https://gitlab.com/groups/gitlab-org/-/epics/9266). +A top-level group can also have one complete +[Security Dashboard and Center](../application_security/security_dashboard/index.md), +[Vulnerability](../application_security/vulnerability_report/index.md#vulnerability-report) and +[Compliance Report](../compliance/compliance_report/index.md), and +[Value stream analytics](../group/value_stream_analytics/index.md). + ## View groups To view groups, on the top bar, select **Main menu > Groups > View all groups**. @@ -65,10 +75,10 @@ This action removes the group. It also adds a background job to delete all proje Specifically: -- In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [GitLab Premium](https://about.gitlab.com/pricing/premium/) or higher tiers, this action adds a background job to mark a group for deletion. By default, the job schedules the deletion 7 days in the future. You can modify this waiting period through the [instance settings](../admin_area/settings/visibility_and_access_controls.md#deletion-protection). +- In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [GitLab Premium or Ultimate tiers](https://about.gitlab.com/pricing/premium/), this action adds a background job to mark a group for deletion. By default, the job schedules the deletion 7 days in the future. You can modify this waiting period through the [instance settings](../admin_area/settings/visibility_and_access_controls.md#deletion-protection). - In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/39504), if the user who sets up the deletion is removed from the group before the -deletion happens, the job is cancelled, and the group is no longer scheduled for deletion. + deletion happens, the job is cancelled, and the group is no longer scheduled for deletion. ## Remove a group immediately **(PREMIUM)** @@ -155,7 +165,7 @@ You can sort members by **Account**, **Access granted**, **Max role**, or **Last 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Members**. -1. Above the list of members, in the upper right, from the **Account** list, select +1. Above the list of members, in the upper-right corner, from the **Account** list, select the criteria to filter by. 1. To switch the sort between ascending and descending, to the right of the **Account** list, select the arrow (**{sort-lowest}** or **{sort-highest}**). @@ -237,6 +247,23 @@ To change this setting for a specific group: To change this setting globally, see [Default project creation protection](../admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). +## Add Group README + +As a group owner or member, you can use a README to provide more information about your team, and invite users to contribute to your projects. +The README is displayed on the group overview page, and can be changed in the group settings. All group members can edit the README. + +Prerequisite: + +- To create the README from the group settings, you must have the Owner role for the group. + +To add a group README: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. In the **Group README** section, select **Add README**. This action creates a new project `gitlab-profile` that contains the `README.md` file. +1. On the prompt for creating a README, select **Create and add README**. You're redirected to the Web IDE, where a README file is created. +1. In the Web IDE, edit and commit the `README.md` file. + ## Change the owner of a group You can change the owner of a group. Each group must always have at least one @@ -297,7 +324,7 @@ To change this setting for a specific group, see [group level default branch pro To change this setting globally, see [initial default branch protection](../project/repository/branches/default.md#instance-level-default-branch-protection). NOTE: -In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../project/repository/branches/default.md#prevent-overrides-of-default-branch-protection). +In [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../project/repository/branches/default.md#prevent-overrides-of-default-branch-protection). ## Use a custom name for the initial branch @@ -348,11 +375,14 @@ If you need to copy a group to a different GitLab instance, When transferring groups, note: - Changing a group's parent can have unintended side effects. See [what happens when a repository path changes](../project/repository/index.md#what-happens-when-a-repository-path-changes). -- You can only transfer groups to groups you manage. +- You must have the Owner role in the source and target group. - You must update your local repositories to point to the new location. - If the immediate parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects change to match the new parent group's visibility. - Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. -- Transfers fail if [packages](../packages/index.md) exist in any of the projects in the group, or in any of its subgroups. +- Transfers fail if [npm packages](../packages/npm_registry/index.md) exist in any of the projects in the group, or in any of its subgroups. +- Existing packages that use a group-level endpoint (Maven, NuGet, PyPI, Composer, and Debian) need to be updated per the package's steps for setting up the group level endpoint. +- Existing package names need to be updated if the package uses an instance level endpoint ([Maven](../packages/maven_repository/index.md#naming-convention), [npm](../packages/npm_registry/index.md#naming-convention), [Conan](../packages/conan_repository/index.md#package-recipe-naming-convention-for-instance-remotes)) and the group was moved to another root level namespace. +- [Maven packages](../packages/maven_repository/index.md#naming-convention) follow a naming convention that prevent installing or publishing the respective package from a group level endpoint after group transfer. - Top-level groups that have a subscription on GitLab.com cannot be transferred. To make the transfer possible, the top-level group's subscription must be removed first. Then the top-level group can be transferred as a subgroup to another top-level group. To transfer a group: @@ -371,6 +401,9 @@ To transfer a group: > - [Instance setting to enable by default added](https://gitlab.com/gitlab-org/gitlab/-/issues/255449) in GitLab 14.2. > - [Instance setting is inherited and enforced when disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. > - [User interface changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352961) in GitLab 15.1. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. +> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) on May 08, 2023. +> - Enabled delayed deletion by default and removed the option to delete immediately [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. [Delayed project deletion](../project/settings/index.md#delayed-project-deletion) is locked and disabled unless the instance-level settings for [deletion protection](../admin_area/settings/visibility_and_access_controls.md#deletion-protection) are enabled for either groups only or groups and projects. @@ -398,8 +431,10 @@ To enable delayed deletion of projects in a group: - (GitLab 15.0 and earlier), **Enforce for all subgroups**. 1. Select **Save changes**. -NOTE: -In GitLab 13.11 and above the group setting for delayed project deletion is inherited by subgroups. As discussed in [Cascading settings](../../development/cascading_settings.md) inheritance can be overridden, unless enforced by an ancestor. +In GitLab 13.11 and later, the group setting for delayed project deletion is inherited by subgroups. As discussed in [Cascading settings](../../development/cascading_settings.md), inheritance can be overridden unless enforced by an ancestor. + +In GitLab 15.11 with the `always_perform_delayed_deletion` feature flag enabled, this setting is removed +and all projects are deleted after the [retention period defined by the admin](../admin_area/settings/visibility_and_access_controls.md#retention-period). This will be the default behavior in GitLab 16.0 and later. ## Disable email notifications @@ -447,6 +482,10 @@ You can export a list of members in a group or subgroup as a CSV. 1. Select **Export as CSV**. 1. After the CSV file has been generated, it is emailed as an attachment to the user that requested it. +The output lists direct members and members inherited from the ancestor groups. +For members with `Minimal Access` in the selected group, their `Max Role` and `Source` are derived from their membership in subgroups. +[Issue 390358](https://gitlab.com/gitlab-org/gitlab/-/issues/390358) tracks the discussion about the group members CSV export list not matching the UI members list. + ## User cap for groups > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330027) in GitLab 14.7. @@ -465,7 +504,7 @@ disabled for the group and its subgroups. Prerequisite: -- You must be assigned the Owner role) for the group. +- You must be assigned the Owner role for the group. To specify a user cap: @@ -640,9 +679,78 @@ To view the merge request approval settings for a group: Support for group-level settings for merge request approval rules is tracked in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/4367). +## Group Code Suggestions **(FREE SAAS)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/405126) in GitLab 15.11. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/408158) from GitLab Ultimate to GitLab Premium in 16.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/410801) from GitLab Premium to GitLab Free in 16.0. + +WARNING: +This feature is in [Beta](../../policy/alpha-beta-support.md#beta). +Code Suggestions use generative AI to suggest code while you're developing. +Due to high demand, this feature will have unscheduled downtime and code suggestions in VS Code may be delayed. +Code Suggestions may produce +[low-quality or incomplete suggestions](../project/repository/code_suggestions.md#model-accuracy-and-quality). +Beta users should read about the [known limitations](../project/repository/code_suggestions.md#known-limitations). +We look forward to hearing your feedback. + +This setting enables users in the group to access [Code Suggestions](../project/repository/code_suggestions.md). +This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) +that belong to the group. + +To enable Code Suggestions for a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Permissions and group features** section. +1. Find the **Code Suggestions** settings. +1. Select **Save changes**. + +## Group Experiment features setting **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404856) in GitLab 16.0. + +You can give all users in the group access to Experiment features. + +WARNING: +[Experiment features](../../policy/alpha-beta-support.md#experiment) may produce unexpected results +(for example, the results might be low-quality, incomplete, incoherent, offensive, or insensitive, +and might include insecure code or failed pipelines). + +This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) +that belong to the group. + +To enable Experiment features for a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Permissions and group features** section. +1. Find the **Experiment features** settings. +1. Select **Save changes**. + +## Group third-party AI features setting **(ULTIMATE SAAS)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404856) in GitLab 16.0. + +WARNING: +These Artifical Intelligence (AI) features use [third-party services](../ai_features.md#data-usage) +and require transmission of data, including personal data. + +You can give all users in the group access to third-party AI features. +This setting [cascades to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) +that belong to the group. + +To enable third-party AI features for a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Permissions and group features** section. +1. Find the **Third-party Artificial Intelligence (AI) features** settings. +1. Select **Save changes**. + ## Group activity analytics **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/alpha-beta-support.md#beta). For a group, you can view how many merge requests, issues, and members were created in the last 90 days. @@ -745,3 +853,16 @@ Administrators can find a user's maximum permissions for a group or project. group = Group.find_by_full_path 'group' user.max_member_access_for_group group.id ``` + +### Unable to remove billable members with badge `Project Invite/Group Invite` + +```plaintext +Members who were invited via a group invitation cannot be removed. You can either remove the entire group, or ask an Owner of the invited group to remove the member. +``` + +This error typically occurs when the user you're trying to remove is part of an external group that has been [shared with one or more of your projects](../project/members/share_project_with_groups.md) or [groups](#share-a-group-with-another-group). To remove the user as a billable member, follow one of the options: + +- Remove the invited group membership from your project or group members page. +- Recommended. Remove the user directly from the invited group, if you have access to the group. + +The feature request to **Update billable_members endpoint to include invited group** is currently being worked on. For more information, see [issue 386583](https://gitlab.com/gitlab-org/gitlab/-/issues/386583) diff --git a/doc/user/group/moderate_users.md b/doc/user/group/moderate_users.md new file mode 100644 index 00000000000..4ec86893524 --- /dev/null +++ b/doc/user/group/moderate_users.md @@ -0,0 +1,48 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Moderate users **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/155) in GitLab 15.8. + +This is the group-level documentation. For self-managed instances, see the [administration documentation](../admin_area/moderate_users.md). + +A group Owner can moderate user access by banning and unbanning users. + +## Unban a user + +To unban a user with the GraphQL API, see [`Mutation.namespaceBanDestroy`](../../api/graphql/reference/index.md#mutationnamespacebandestroy). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo on unbanning a user at the group level, see [Namespace level ban - Unbanning a user](https://www.youtube.com/watch?v=mTQVbP3MQrs). + +Prerequisites: + +- In the top-level group, you must have the Owner role. + +To unban a user: + +1. Go to the top-level group. +1. On the left sidebar, select **Group information > Members**. +1. Select the **Banned** tab. +1. For the account you want to unban, select **Unban**. + +## Ban a user + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo on banning a user at the group level, see [Namespace level ban - Banning a user](https://youtu.be/1rbi1uEJmOI). + +Prerequisites: + +- In the top-level group, you must have the Owner role. +- In the top-level group, if the user you want to ban has the Owner role, you must [demote the user](manage.md#change-the-owner-of-a-group). + +To manually ban a user: + +1. Go to the top-level group. +1. On the left sidebar, select **Group information > Members**. +1. Next to the member you want to ban, select the vertical ellipsis (**{ellipsis_v}**). +1. From the dropdown list, select **Ban member**. diff --git a/doc/user/group/reporting/git_abuse_rate_limit.md b/doc/user/group/reporting/git_abuse_rate_limit.md index 14b188e1204..d5c44f4e245 100644 --- a/doc/user/group/reporting/git_abuse_rate_limit.md +++ b/doc/user/group/reporting/git_abuse_rate_limit.md @@ -11,13 +11,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. On GitLab.com, this feature is available. -Git abuse rate limiting is a feature to automatically ban users who download or clone more than a specified number of repositories in a group or any of its subgroups within a given time frame. Banned users cannot access the main group or any of its non-public subgroups via HTTP or SSH. Access to unrelated groups is unaffected. +This is the group-level documentation. For self-managed instances, see the [administration documentation](../../admin_area/reporting/git_abuse_rate_limit.md). -If the `limit_unique_project_downloads_per_namespace_user` feature flag is enabled, all users with the Owner role for the main group receive an email when a user is about to be banned. +Git abuse rate limiting is a feature to automatically ban users who download, clone, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. -If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, users with the Owner role for the main group are still notified. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. +Git abuse rate limiting does not apply to top-level group owners, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). -If automatic banning is enabled, users with the Owner role for the main group receive an email when a user is about to be banned, and the user is automatically banned from the group and its subgroups. +## Automatic ban notifications + +If the `limit_unique_project_downloads_per_namespace_user` feature flag is enabled, selected users receive an email when a user is about to be banned. + +If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. + +If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the group and its subgroups. ## Configure Git abuse rate limiting @@ -26,29 +32,10 @@ If automatic banning is enabled, users with the Owner role for the main group re 1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled. 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled. 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned. + 1. Add up to `100` users to the **Send notifications to** field. You must select at least one user. All users with the Owner role for the main group are selected by default. 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. 1. Select **Save changes**. -## Unban a user - -Prerequisites: - -- You must have the Owner role. - -1. On the left sidebar, select **Group information > Members**. -1. Select the **Banned** tab. -1. For the account you want to unban, select **Unban**. - -## Ban a user - -> [Introduced](https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/155) in GitLab 15.8. - -Prerequisites: - -- You must have the Owner role. - -To manually ban a user: +## Related topics -1. On the left sidebar, select **Group information > Members**. -1. Next to the member you want to ban, select the vertical ellipsis (**{ellipsis_v}**). -1. From the dropdown list, select **Ban member**. +- [Ban and unban users](../moderate_users.md). diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index 9971457f2ac..c2a4d8175b3 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -42,7 +42,7 @@ To see the latest code coverage for each project in your group: 1. In the **Latest test coverage results** section, from the **Select projects** dropdown list, choose the projects you want to check. You can download code coverage data for specific projects using -[code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). +[code coverage history](../../../ci/testing/code_coverage.md#view-history-of-project-code-coverage). ## Download historic test coverage data diff --git a/doc/user/group/saml_sso/example_saml_config.md b/doc/user/group/saml_sso/example_saml_config.md index 7778648e985..524a5d5a9bd 100644 --- a/doc/user/group/saml_sso/example_saml_config.md +++ b/doc/user/group/saml_sso/example_saml_config.md @@ -58,6 +58,8 @@ Attribute mapping: NOTE: Using the **Group ID** source attribute requires users to enter the group ID or object ID when configuring SAML group links. If available, use the **sAMAccountName** source attribute for the friendly group name instead. +[Azure AD limits the number of groups that can be sent in a SAML response to 150](https://support.esri.com/en-us/knowledge-base/000022190'). If a user is a member of more than 150 groups, Azure does not include that user's group claim in the SAML response. + ## Google Workspace Basic SAML app configuration: diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md index 27482893bd6..ee59eeb98db 100644 --- a/doc/user/group/saml_sso/group_sync.md +++ b/doc/user/group/saml_sso/group_sync.md @@ -22,10 +22,7 @@ For a demo of Group Sync using Azure, see [Demo: SAML Group Sync](https://youtu. ## Configure SAML Group Sync NOTE: -You must include the SAML configuration block on all Sidekiq nodes in addition to Rails application nodes if you: - -- Use SAML Group Sync. -- Have multiple GitLab nodes, for example in a distributed or highly available architecture. +You must include the SAML configuration block on all Sidekiq nodes in addition to Rails application nodes if you use SAML Group Sync and have multiple GitLab nodes, for example in a distributed or highly available architecture. NOTE: SAML Group Sync is only supported for the [SAML provider named `saml`](../../../integration/saml.md#configure-gitlab-to-use-multiple-saml-idps). @@ -107,11 +104,37 @@ Users granted: - A lower or the same role with Group Sync are displayed as having [inherited membership](../../project/members/index.md#display-inherited-members) of the group. +SAML group membership is evaluated each time a user signs in. + +### Global SAML group memberships lock **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386390) in GitLab 15.10. + +GitLab administrators can use the global SAML group memberships lock to prevent group members from inviting new members to subgroups that have their membership synchronized with SAML Group Links. + +Global group memberships lock only applies to subgroups of a top-level group where SAML Group Links synchronization is configured. No user can modify the +membership of a top-level group configured for SAML Group Links synchronization. + +When global group memberships lock is enabled: + +- Only an administrator can manage memberships of any group including access levels. +- Users cannot: + - Share a project with other groups. + - Invite members to a project created in a group. + +To enable global group memberships lock: + +1. [Configure SAML](../../../integration/saml.md) for your self-managed GitLab instance. +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Ensure the **Lock memberships to SAML synchronization** checkbox is selected. + ### Automatic member removal After a group sync, users who are not members of a mapped SAML group are removed from the group. On GitLab.com, users in the top-level group are assigned the -[default membership role](index.md#role) instead of being removed. +default membership role instead of being removed. For example, in the following diagram: diff --git a/doc/user/group/saml_sso/img/Azure-manage-group-claims.png b/doc/user/group/saml_sso/img/Azure-manage-group-claims.png Binary files differindex 2ff24733282..b08b6ab4907 100644 --- a/doc/user/group/saml_sso/img/Azure-manage-group-claims.png +++ b/doc/user/group/saml_sso/img/Azure-manage-group-claims.png diff --git a/doc/user/group/saml_sso/img/group_saml_configuration_information.png b/doc/user/group/saml_sso/img/group_saml_configuration_information.png Binary files differdeleted file mode 100644 index e03c50ceb01..00000000000 --- a/doc/user/group/saml_sso/img/group_saml_configuration_information.png +++ /dev/null diff --git a/doc/user/group/saml_sso/img/group_saml_settings_v13_12.png b/doc/user/group/saml_sso/img/group_saml_settings_v13_12.png Binary files differdeleted file mode 100644 index 2271f281655..00000000000 --- a/doc/user/group/saml_sso/img/group_saml_settings_v13_12.png +++ /dev/null diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index e650b2dd130..a54b3fea53e 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -8,249 +8,80 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Introduced in GitLab 11.0. -This page describes SAML for groups. For instance-wide SAML on self-managed GitLab instances, see [SAML SSO for self-managed GitLab instances](../../../integration/saml.md). -[View the differences between SaaS and Self-Managed Authentication and Authorization Options](../../../administration/auth/index.md#saas-vs-self-managed-comparison). +Users can sign in to GitLab through their SAML identity provider. -SAML on GitLab.com allows users to sign in through their SAML identity provider. If the user is not already a member, the sign-in process automatically adds the user to the appropriate group. +[SCIM](scim_setup.md) synchronizes users with the group on GitLab.com. -User synchronization of SAML SSO groups is supported through [SCIM](scim_setup.md). SCIM supports adding and removing users from the GitLab group automatically. -For example, if you remove a user from the SCIM app, SCIM removes that same user from the GitLab group. +- When you add or remove a user from the SCIM app, SCIM adds or removes the user + from the GitLab group. +- If the user is not already a group member, the user is added to the group as part of the sign-in process. -SAML SSO is only configurable at the top-level group. +You can configure SAML SSO for the top-level group only. -If required, you can find [a glossary of common terms](../../../integration/saml.md#glossary-of-common-terms). +## Set up your identity provider -## Configure your identity provider - -1. Find the information in GitLab required for configuration: - 1. On the top bar, select **Main menu > Groups** and find your group. - 1. On the left sidebar, select **Settings > SAML SSO**. - 1. Note the **Assertion consumer service URL**, **Identifier**, and **GitLab single sign-on URL**. -1. Configure your SAML identity provider app using the noted details. - Alternatively, GitLab provides a [metadata XML configuration](#metadata-configuration). - See [specific identity provider documentation](#providers) for more details. -1. Configure the SAML response to include a [NameID](#nameid) that uniquely identifies each user. -1. Configure the required [user attributes](#user-attributes), ensuring you include the user's email address. -1. While the default is enabled for most SAML providers, ensure the app is set to have service provider - initiated calls to link existing GitLab accounts. -1. Once the identity provider is set up, move on to [configuring GitLab](#configure-gitlab). - -![Issuer and callback for configuring SAML identity provider with GitLab.com](img/group_saml_configuration_information.png) - -If your account is the only owner in the group after SAML is set up, you can't unlink the account. To [unlink the account](#unlinking-accounts), -set up another user as a group owner. - -### NameID - -GitLab.com uses the SAML NameID to identify users. The NameID element: - -- Is a required field in the SAML response. -- Must be unique to each user. -- Must be a persistent value that never changes, such as a randomly generated unique user ID. -- Is case sensitive. The NameID must match exactly on subsequent login attempts, so should not rely on user input that could change between upper and lower case. -- Should not be an email address or username. We strongly recommend against these as it's hard to - guarantee it doesn't ever change, for example, when a person's name changes. Email addresses are - also case-insensitive, which can result in users being unable to sign in. - -The relevant field name and recommended value for supported providers are in the [provider specific notes](#providers). - -WARNING: -Once users have signed into GitLab using the SSO SAML setup, changing the `NameID` breaks the configuration and potentially locks users out of the GitLab group. - -#### NameID Format - -We recommend setting the NameID format to `Persistent` unless using a field (such as email) that requires a different format. -Most NameID formats can be used, except `Transient` due to the temporary nature of this format. - -### User attributes - -To create users with the correct information for improved [user access and management](#user-access-and-management), -the user's details must be passed to GitLab as attributes in the SAML assertion. At a minimum, the user's email address -must be specified as an attribute named `email` or `mail`. - -You can configure the following attributes with GitLab.com Group SAML: +The SAML standard means that you can use a wide range of identity providers with GitLab. Your identity provider might have relevant documentation. It can be generic SAML documentation or specifically targeted for GitLab. -- `username` or `nickname`. We recommend you configure only one of these. -- The [attributes available](../../../integration/saml.md#configure-assertions) to self-managed GitLab instances. +When setting up your identity provider, use the following provider-specific documentation +to help avoid common issues and as a guide for terminology used. -### Metadata configuration +For identity providers not listed, you can refer to the [instance SAML notes on configuring an identity provider](../../../integration/saml.md#configure-saml-on-your-idp) +for additional guidance on information your provider may require. -GitLab provides metadata XML that can be used to configure your identity provider. +GitLab provides the following information for guidance only. +If you have any questions on configuring the SAML app, contact your provider's support. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > SAML SSO**. -1. Copy the provided **GitLab metadata URL**. -1. Follow your identity provider's documentation and paste the metadata URL when it's requested. +If you are having issues setting up your identity provider, see the +[troubleshooting documentation](#troubleshooting). -## Configure GitLab +### Azure -After you set up your identity provider to work with GitLab, you must configure GitLab to use it for authentication: +To set up SSO with Azure as your identity provider: -1. On the top bar, select **Main menu > Groups** and find your group. +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > SAML SSO**. -1. Find the SSO URL from your identity provider and enter it the **Identity provider single sign-on URL** field. -1. Find and enter the fingerprint for the SAML token signing certificate in the **Certificate** field. -1. Select the access level to be applied to newly added users in the **Default membership role** field. The default access level is 'Guest'. -1. Select the **Enable SAML authentication for this group** checkbox. -1. Select the **Save changes** button. - -![Group SAML Settings for GitLab.com](img/group_saml_settings_v13_12.png) +1. Note the information on this page. +1. Go to Azure and [follow the instructions for configuring SSO for an application](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/add-application-portal-setup-sso). The following GitLab settings correspond to the Azure fields. -NOTE: -The certificate [fingerprint algorithm](../../../integration/saml.md#configure-saml-on-your-idp) must be in SHA1. When configuring the identity provider (such as [Google Workspace](#set-up-google-workspace)), use a secure signature algorithm. + | GitLab setting | Azure field | + | -----------------------------------------| ---------------------------------------------- | + | **Identifier** | **Identifier (Entity ID)** | + | **Assertion consumer service URL** | **Reply URL (Assertion Consumer Service URL)** | + | **GitLab single sign-on URL** | **Sign on URL** | + | **Identity provider single sign-on URL** | **Login URL** | + | **Certificate fingerprint** | **Thumbprint** | -### Additional configuration information +1. You should set the following attributes: + - **Unique User Identifier (Name identifier)** to `user.objectID`. + - **nameid-format** to `persistent`. For more information, see how to [manage user SAML identity](#manage-user-saml-identity). + - **Additional claims** to [supported attributes](#user-attributes). -Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. -For more information, start with your identity provider's documentation. Look for their options and examples to see how they configure SAML. This can provide hints on what you need to configure GitLab to work with these providers. - -It can also help to look at our [more detailed docs for self-managed GitLab](../../../integration/saml.md). -SAML configuration for GitLab.com is mostly the same as for self-managed instances. -However, self-managed GitLab instances use a configuration file that supports more options as described in the external [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). -Internally that uses the [`ruby-saml` library](https://github.com/onelogin/ruby-saml), so we sometimes check there to verify low level details of less commonly used options. - -It can also help to compare the XML response from your provider with our [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). - -### SSO enforcement - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/292811) in GitLab 13.8, with an updated timeout experience. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/211962) in GitLab 13.8 with allowing group owners to not go through SSO. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in GitLab 13.11 with enforcing open SSO session to use Git if this setting is switched on. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/339888) in GitLab 14.7 to not enforce SSO checks for Git activity originating from CI/CD jobs. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Disabled on GitLab.com. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/375788) in GitLab 15.8 by enabling transparent SSO by default on GitLab.com. - -FLAG: -On self-managed GitLab, transparent SSO enforcement is unavailable. An -[issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/382917) to add -transparent SSO enforcement to self-managed GitLab. -On GitLab.com, transparent SSO enforcement is available by default. To turn off -transparent SSO, ask a support or production team to enable the -`transparent_sso_enforcement_override` feature flag for a specific customer -group. - -#### Transparent SSO enforcement - -By default, transparent SSO enforcement is enabled in GitLab.com. This means SSO is enforced: - -- When users access groups and projects in the organization's - group hierarchy. Users can view other groups and projects without SSO sign in. -- For each user with an existing SAML identity. - -When transparent SSO enforcement is enabled, users: - -- Are not prompted to sign in through SSO on each visit. GitLab checks - whether a user has authenticated through SSO. If the user last signed in more - than 24 hours ago, GitLab prompts the user to sign in again through SSO. -- Without SAML identities are not required to use SSO unless **Enforce - SSO-only authentication for web activity for this group** is enabled. - -A user has a SAML identity if one or both of the following are true: - -- They have signed in to GitLab by using their GitLab group's single sign-on URL. -- They were provisioned by SCIM. - -With transparent SSO enabled, SSO is enforced as follows: - -| Project/Group visibility | Enforce SSO setting | Member with identity | Member without identity | Non-member or not signed in | -|--------------------------|---------------------|--------------------| ------ |------------------------------| -| Private | Off | Enforced | Not enforced | No access | -| Private | On | Enforced | Enforced | No access | -| Public | Off | Enforced | Not enforced | Not enforced | -| Public | On | Enforced | Enforced | Not enforced | - -An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity. - -#### SSO-only for web activity enforcement - -When the **Enforce SSO-only authentication for web activity for this group** option is enabled: - -- All users must access GitLab by using their GitLab group's single sign-on URL - to access group resources, regardless of whether they have an existing SAML - identity. -- SSO is enforced when users access groups and projects in the organization's - group hierarchy. Users can view other groups and projects without SSO sign in. -- Users cannot be added as new members manually. -- Users with the Owner role can use the standard sign in process to make - necessary changes to top-level group settings. - -SSO enforcement for web activity has the following effects when enabled: - -- For groups, users cannot share a project in the group outside the top-level - group, even if the project is forked. -- For Git activity over SSH and HTTPS, users must have at least one active - session signed-in through SSO before they can push to or - pull from a GitLab repository. -- Git activity originating from CI/CD jobs do not have the SSO check enforced. -- Credentials that are not tied to regular users (for example, project and group - access tokens, and deploy keys) do not have the SSO check enforced. -- Users must be signed-in through SSO before they can pull images using the - [Dependency Proxy](../../packages/dependency_proxy/index.md). -- When the **Enforce SSO-only authentication for Git and Dependency Proxy - activity for this group** option is enabled, any API endpoint that involves - Git activity is under SSO enforcement. For example, creating or deleting a - branch, commit, or tag. - -When SSO for web activity is enforced, non-SSO group members do not lose access -immediately. If the user: - -- Has an active session, they can continue accessing the group for up to 24 - hours until the identity provider session times out. -- Is signed out, they cannot access the group after being removed from the - identity provider. - -## Providers - -The SAML standard means that you can use a wide range of identity providers with GitLab. Your identity provider might have relevant documentation. It can be generic SAML documentation or specifically targeted for GitLab. - -When [configuring your identity provider](#configure-your-identity-provider), consider the notes below for specific providers to help avoid common issues and as a guide for terminology used. - -For providers not listed below, you can refer to the [instance SAML notes on configuring an identity provider](../../../integration/saml.md#configure-saml-on-your-idp) -for additional guidance on information your identity provider may require. - -GitLab provides the following information for guidance only. -If you have any questions on configuring the SAML app, contact your provider's support. - -### Set up Azure - -Follow the Azure documentation on [configuring single sign-on to applications](https://learn.microsoft.com/en-us/azure/active-directory/manage-apps/add-application-portal-setup-sso), and use the following notes when needed. +1. Optional. If you use [Group Sync](group_sync.md), customize the name of the + group claim to match the required attribute. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azure Using SAML SSO for Groups Demo](https://youtu.be/24-ZxmTeEBU). -The video is outdated in regard to objectID mapping and you should follow the [SCIM documentation](scim_setup.md#configure-azure-active-directory). - -| GitLab Setting | Azure Field | -| ------------------------------------ | ------------------------------------------ | -| Identifier | Identifier (Entity ID) | -| Assertion consumer service URL | Reply URL (Assertion Consumer Service URL) | -| GitLab single sign-on URL | Sign on URL | -| Identity provider single sign-on URL | Login URL | -| Certificate fingerprint | Thumbprint | - -You should set the following attributes: - -- **Unique User Identifier (Name identifier)** to `user.objectID`. -- **nameid-format** to persistent. -- Additional claims to [supported attributes](#user-attributes). +View a demo of [SCIM provisioning on Azure using SAML SSO for groups](https://youtu.be/24-ZxmTeEBU). The `objectID` mapping is outdated in this video. Follow the [SCIM documentation](scim_setup.md#configure-azure-active-directory) instead. -If using [Group Sync](#group-sync), customize the name of the group claim to match the required attribute. +For more information, see an [example configuration page](example_saml_config.md#azure-active-directory). -See our [example configuration page](example_saml_config.md#azure-active-directory). +### Google Workspace -### Set up Google Workspace +To set up Google Workspace as your identity provider: -1. [Set up SSO with Google as your identity provider](https://support.google.com/a/answer/6087519?hl=en). - The following GitLab settings correspond to the Google Workspace fields. +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. +1. Follow the instructions for [setting up SSO with Google as your identity provider](https://support.google.com/a/answer/6087519?hl=en). The following GitLab settings correspond to the Google Workspace fields. - | GitLab setting | Google Workspace field | - |:-------------------------------------|:-----------------------| - | Identifier | **Entity ID** | - | Assertion consumer service URL | **ACS URL** | - | GitLab single sign-on URL | **Start URL** | - | Identity provider single sign-on URL | **SSO URL** | + | GitLab setting | Google Workspace field | + |:-----------------------------------------|:-----------------------| + | **Identifier** | **Entity ID** | + | **Assertion consumer service URL** | **ACS URL** | + | **GitLab single sign-on URL** | **Start URL** | + | **Identity provider single sign-on URL** | **SSO URL** | 1. Google Workspace displays a SHA256 fingerprint. To retrieve the SHA1 fingerprint required by GitLab to [configure SAML](#configure-gitlab): @@ -262,89 +93,151 @@ See our [example configuration page](example_saml_config.md#azure-active-directo ``` 1. Set these values: - - For **Primary email**: `email` - - For **First name**: `first_name` - - For **Last name**: `last_name` - - For **Name ID format**: `EMAIL` - - For **NameID**: `Basic Information > Primary email` + - For **Primary email**: `email`. + - For **First name**: `first_name`. + - For **Last name**: `last_name`. + - For **Name ID format**: `EMAIL`. + - For **NameID**: `Basic Information > Primary email`. + For more information, see [manage user SAML identity](#manage-user-saml-identity). + +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. On the GitLab SAML SSO page, when you select **Verify SAML Configuration**, disregard the warning that recommends setting the **NameID** format to `persistent`. -For details, see the [example configuration page](example_saml_config.md#google-workspace). - -### Set up Okta +For more information, see an [example configuration page](example_saml_config.md#google-workspace). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). +View a demo of [how to configure SAML with Google Workspaces and set up Group Sync](https://youtu.be/NKs0FSQVfCY). + +### Okta + +To set up SSO with Okta as your identity provider: + +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. +1. Follow the instructions for [setting up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/). -1. [Set up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/). The following GitLab settings correspond to the Okta fields. - | GitLab setting | Okta field | - | ------------------------------------ | ---------------------------------------------------------- | - | Identifier | **Audience URI** | - | Assertion consumer service URL | **Single sign-on URL** | - | GitLab single sign-on URL | **Login page URL** (under **Application Login Page** settings) | - | Identity provider single sign-on URL | **Identity Provider Single Sign-On URL** | + | GitLab setting | Okta field | + | ---------------------------------------- | -------------------------------------------------------------- | + | **Identifier** | **Audience URI** | + | **Assertion consumer service URL** | **Single sign-on URL** | + | **GitLab single sign-on URL** | **Login page URL** (under **Application Login Page** settings) | + | **Identity provider single sign-on URL** | **Identity Provider Single Sign-On URL** | 1. Under the Okta **Single sign-on URL** field, select the **Use this for Recipient URL and Destination URL** checkbox. 1. Set these values: - - For **Application username (NameID)**: **Custom** `user.getInternalProperty("id")` - - For **Name ID Format**: `Persistent` + - For **Application username (NameID)**: **Custom** `user.getInternalProperty("id")`. + - For **Name ID Format**: `Persistent`. For more information, see [manage user SAML identity](#manage-user-saml-identity). + +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. The Okta GitLab application available in the App Catalog only supports [SCIM](scim_setup.md). Support for SAML is proposed in [issue 216173](https://gitlab.com/gitlab-org/gitlab/-/issues/216173). -### Set up OneLogin +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). + +For more information, see an [example configuration page](example_saml_config.md#okta) + +### OneLogin OneLogin supports its own [GitLab (SaaS) application](https://onelogin.service-now.com/support?id=kb_article&sys_id=92e4160adbf16cd0ca1c400e0b961923&kb_category=50984e84db738300d5505eea4b961913). +To set up OneLogin as your identity provider: + +1. In GitLab, on the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Note the information on this page. 1. If you use the OneLogin generic [SAML Test Connector (Advanced)](https://onelogin.service-now.com/support?id=kb_article&sys_id=b2c19353dbde7b8024c780c74b9619fb&kb_category=93e869b0db185340d5505eea4b961934), you should [use the OneLogin SAML Test Connector](https://onelogin.service-now.com/support?id=kb_article&sys_id=93f95543db109700d5505eea4b96198f). The following GitLab settings correspond to the OneLogin fields: - | GitLab setting | OneLogin field | - | ------------------------------------------------ | -------------------------------- | - | Identifier | **Audience** | - | Assertion consumer service URL | **Recipient** | - | Assertion consumer service URL | **ACS (Consumer) URL** | - | Assertion consumer service URL (escaped version) | **ACS (Consumer) URL Validator** | - | GitLab single sign-on URL | **Login URL** | - | Identity provider single sign-on URL | **SAML 2.0 Endpoint** | + | GitLab setting | OneLogin field | + | ---------------------------------------------------- | -------------------------------- | + | **Identifier** | **Audience** | + | **Assertion consumer service URL** | **Recipient** | + | **Assertion consumer service URL** | **ACS (Consumer) URL** | + | **Assertion consumer service URL (escaped version)** | **ACS (Consumer) URL Validator** | + | **GitLab single sign-on URL** | **Login URL** | + | **Identity provider single sign-on URL** | **SAML 2.0 Endpoint** | -1. For **NameID**, use `OneLogin ID`. +1. For **NameID**, use `OneLogin ID`. For more information, see [manage user SAML identity](#manage-user-saml-identity). -### Change the SAML app +1. Make sure the identity provider is set to have provider-initiated calls + to link existing GitLab accounts. -To change the SAML app used for sign in: +### Use metadata -- If the NameID is not identical in both the existing and new SAML apps, users must: - 1. [Unlink the current SAML identity](#unlinking-accounts). - 1. [Link their identity](#user-access-and-management) to the new SAML app. -- If the NameID is identical, no change is required. +To configure some identity providers, you need a GitLab metadata URL. +To find this URL: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Copy the provided **GitLab metadata URL**. +1. Follow your identity provider's documentation and paste the metadata URL when it's requested. -### Migrate to a different SAML provider +Check your identity provider's documentation to see if it supports the GitLab metadata URL. -You can migrate to a different SAML provider. During the migration process users will not be able to access any of the SAML groups. -To mitigate this, you can disable [SSO enforcement](#sso-enforcement). +### Manage the identity provider -To migrate SAML providers: +After you have set up your identity provider, you can: -1. [Configure](#configure-your-identity-provider) the group with the new identity provider SAML app. -1. Ask users to [unlink their account from the group](#unlinking-accounts). -1. Ask users to [link their account to the new SAML app](#linking-saml-to-your-existing-gitlabcom-account). +- Change the identity provider. +- Change email domains. -### Change email domains +#### Change the identity provider -To migrate users to a new email domain, users must: +You can change to a different identity provider. During the change process, +users cannot access any of the SAML groups. To mitigate this, you can disable +[SSO enforcement](#sso-enforcement). + +To change identity providers: + +1. [Configure](#set-up-your-identity-provider) the group with the new identity provider. +1. Optional. If the **NameID** is not identical, [change the **NameID** for users](#manage-user-saml-identity). + +#### Change email domains + +To migrate users to a new email domain, tell users to: 1. Add their new email as the primary email to their accounts and verify it. -1. [Unlink their account from the group](#unlinking-accounts). -1. [Link their account to the group](#linking-saml-to-your-existing-gitlabcom-account). -1. (Optional) Remove their old email from the account. +1. Optional. Remove their old email from the account. + +If the **NameID** is configured with the email address, [change the **NameID** for users](#manage-user-saml-identity). + +## Configure GitLab + +After you set up your identity provider to work with GitLab, you must configure GitLab to use it for authentication: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > SAML SSO**. +1. Complete the fields: + - In the **Identity provider single sign-on URL** field, enter the SSO URL from your identity provider. + - In the **Certificate fingerprint** field, enter the fingerprint for the SAML token signing certificate. +1. In the **Default membership role** field, select the role to assign to new users. + The default role is **Guest**. In [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/214523) + and later, group owners can set a default membership role other than **Guest**. + To do so, [configure the SAML SSO for the group](#configure-gitlab). That role + becomes the starting role of all users added to the group. +1. Select the **Enable SAML authentication for this group** checkbox. +1. Optional. Select: + - **Enforce SSO-only authentication for web activity for this group**. + - **Enforce SSO-only authentication for Git activity for this group**. + For more information, see the [SSO enforcement documentation](#sso-enforcement). +1. Select **Save changes**. + +NOTE: +The certificate [fingerprint algorithm](../../../integration/saml.md#configure-saml-on-your-idp) must be in SHA1. When configuring the identity provider (such as [Google Workspace](#google-workspace)), use a secure signature algorithm. + +If you are having issues configuring GitLab, see the [troubleshooting documentation](#troubleshooting). ## User access and management @@ -362,53 +255,108 @@ When a user tries to sign in with Group SSO, GitLab attempts to find or create a - Create a new account with another email address. - Sign-in to their existing account to link the SAML identity. -### Linking SAML to your existing GitLab.com account +### User attributes + +You can pass user information to GitLab as attributes in the SAML assertion. + +- The user's email address can be an **email** or **mail** attribute. +- The username can be either a **username** or **nickname** attribute. You should specify only + one of these. + +For more information, see the [attributes available for self-managed GitLab instances](../../../integration/saml.md#configure-assertions). + +### Link SAML to your existing GitLab.com account > **Remember me** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/121569) in GitLab 15.7. To link SAML to your existing GitLab.com account: -1. Sign in to your GitLab.com account. [Reset your password](https://gitlab.com/users/password/new) if necessary. -1. Locate and visit the **GitLab single sign-on URL** for the group you're signing in to. A group owner can find this on the group's **Settings > SAML SSO** page. If the sign-in URL is configured, users can connect to the GitLab app from the identity provider. -1. Optional. Select the **Remember me** checkbox to stay signed in to GitLab for 2 weeks. You may still be asked to re-authenticate with your SAML provider - more frequently. +1. Sign in to your GitLab.com account. [Reset your password](https://gitlab.com/users/password/new) + if necessary. +1. Locate and visit the **GitLab single sign-on URL** for the group you're signing + in to. A group owner can find this on the group's **Settings > SAML SSO** page. + If the sign-in URL is configured, users can connect to the GitLab app from the identity provider. +1. Optional. Select the **Remember me** checkbox to stay signed in to GitLab for 2 weeks. + You may still be asked to re-authenticate with your SAML provider more frequently. 1. Select **Authorize**. 1. Enter your credentials on the identity provider if prompted. -1. You are then redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. +1. You are then redirected back to GitLab.com and should now have access to the group. + In the future, you can use SAML to sign in to GitLab.com. + +If a user is already a member of the group, linking the SAML identity does not +change their role. -On subsequent visits, you should be able to go [sign in to GitLab.com with SAML](#signing-in-to-gitlabcom-with-saml) or by visiting links directly. If the **enforce SSO** option is turned on, you are then redirected to sign in through the identity provider. +On subsequent visits, you should be able to [sign in to GitLab.com with SAML](#sign-in-to-gitlabcom-with-saml) +or by visiting links directly. If the **enforce SSO** option is turned on, you +are then redirected to sign in through the identity provider. -### Signing in to GitLab.com with SAML +### Sign in to GitLab.com with SAML 1. Sign in to your identity provider. 1. From the list of apps, select the "GitLab.com" app. (The name is set by the administrator of the identity provider.) 1. You are then signed in to GitLab.com and redirected to the group. -### Change NameID for one or more users +### Manage user SAML identity > Update of SAML identities using the SAML API [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227841) in GitLab 15.5. -Group owners can update the SAML identities for their group members using the [SAML API](../../../api/saml.md). +GitLab.com uses the SAML **NameID** to identify users. The **NameID** is: + +- A required field in the SAML response. +- Case sensitive. + +The **NameID** must: + +- Be unique to each user. +- Be a persistent value that never changes, such as a randomly generated unique user ID. +- Match exactly on subsequent sign-in attempts, so it should not rely on user input + that could change between upper and lower case. + +The **NameID** should not be an email address or username because: + +- Email addresses and usernames are more likely to change over time. For example, + when a person's name changes. +- Email addresses are case-insensitive, which can result in users being unable to + sign in. + +The **NameID** format must be `Persistent`, unless you are using a field, like email, that +requires a different format. You can use any format except `Transient`. + +#### Change user **NameID** + +Group owners can use the [SAML API](../../../api/saml.md#update-extern_uid-field-for-a-saml-identity) to change their group members' **NameID** and update their SAML identities . + +If [SCIM](scim_setup.md) is configured, group owners can update the SCIM identities using the [SCIM API](../../../api/scim.md#update-extern_uid-field-for-a-scim-identity). Alternatively, ask the users to reconnect their SAML account. -1. Ask relevant users to [unlink their account from the group](#unlinking-accounts). -1. Ask relevant users to [link their account to the new SAML app](#linking-saml-to-your-existing-gitlabcom-account). +1. Ask relevant users to [unlink their account from the group](#unlink-accounts). +1. Ask relevant users to [link their account to the new SAML app](#link-saml-to-your-existing-gitlabcom-account). + +WARNING: +After users have signed into GitLab using SSO SAML, changing the **NameID** value +breaks the configuration and could lock users out of the GitLab group. + +For more information on the recommended value and format for specific identity +providers, see [set up your identity provider](#set-up-your-identity-provider). ### Configure user settings from SAML response > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263661) in GitLab 13.7. GitLab allows setting certain user attributes based on values from the SAML response. -Existing users will have these attributes updated if the user was originally -provisioned by the group. Users are provisioned by the group when the account was -created via [SCIM](scim_setup.md) or by first sign-in with SAML SSO for GitLab.com groups. +An existing user's attributes are updated from the SAML response values if that +user was originally provisioned by the group. Users are provisioned by the group +when the account was created either: + +- Through [SCIM](scim_setup.md). +- By first sign-in with SAML SSO for GitLab.com groups. #### Supported user attributes -- `can_create_group` - 'true' or 'false' to indicate whether the user can create +- **can_create_group** - `true` or `false` to indicate whether the user can create new groups. Default is `true`. -- `projects_limit` - The total number of personal projects a user can create. +- **projects_limit** - The total number of personal projects a user can create. A value of `0` means the user cannot create new projects in their personal namespace. Default is `10000`. @@ -446,7 +394,7 @@ convert the information to XML. An example SAML response is shown here. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238461) in GitLab 15.4. By default, users provisioned with SAML or SCIM are sent a verification email to verify their identity. Instead, you can -[configure GitLab with a custom domain](../../project/pages/custom_domains_ssl_tls_certification/index.md) and GitLab +[configure GitLab with a custom domain](../../enterprise_user/index.md#set-up-a-verified-domain) and GitLab automatically confirms user accounts. Users still receive an [enterprise user](../../enterprise_user/index.md) welcome email. Confirmation is bypassed for users: @@ -454,35 +402,25 @@ bypassed for users: - That are provisioned with SAML or SCIM. - That have an email address that belongs to the verified domain. -### Role - -Starting from [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/214523), group owners can set a -"Default membership role" other than Guest. To do so, [configure the SAML SSO for the group](#configure-gitlab). -That role becomes the starting access level of all users added to the group. - -Existing members with appropriate privileges can promote or demote users, as needed. - -If a user is already a member of the group, linking the SAML identity does not change their role. - -Users given a "minimal access" role have [specific restrictions](../../permissions.md#users-with-minimal-access). - -### Blocking access +### Block user access To rescind a user's access to the group when only SAML SSO is configured, either: - Remove (in order) the user from: 1. The user data store on the identity provider or the list of users on the specific app. 1. The GitLab.com group. -- Use Group Sync at the top-level of your group to [automatically remove the user](group_sync.md#automatic-member-removal). +- Use [Group Sync](group_sync.md#automatic-member-removal) at the top-level of + your group with the default role set to [minimal access](../../permissions.md#users-with-minimal-access) + to automatically block access to all resources in the group. To rescind a user's access to the group when also using SCIM, refer to [Remove access](scim_setup.md#remove-access). -### Unlinking accounts +### Unlink accounts Users can unlink SAML for a group from their profile page. This can be helpful if: - You no longer want a group to be able to sign you in to GitLab.com. -- Your SAML NameID has changed and so GitLab can no longer find your user. +- Your SAML **NameID** has changed and so GitLab can no longer find your user. WARNING: Unlinking an account removes all roles assigned to that user in the group. @@ -499,14 +437,106 @@ For example, to unlink the `MyOrg` account: 1. On the left sidebar, select **Account**. 1. In the **Service sign-in** section, select **Disconnect** next to the connected account. -## Group Sync +## SSO enforcement -For information on automatically managing GitLab group membership, see [SAML Group Sync](group_sync.md). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/292811) in GitLab 13.8, with an updated timeout experience. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/211962) in GitLab 13.8 with allowing group owners to not go through SSO. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in GitLab 13.11 with enforcing open SSO session to use Git if this setting is switched on. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/339888) in GitLab 14.7 to not enforce SSO checks for Git activity originating from CI/CD jobs. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Disabled on GitLab.com. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/375788) in GitLab 15.8 by enabling transparent SSO by default on GitLab.com. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/389562) in GitLab 15.10. Feature flag `transparent_sso_enforcement` removed. + +On GitLab.com, SSO is enforced: + +- When SAML SSO is enabled. +- For users with an existing SAML identity when accessing groups and projects in the organization's + group hierarchy. Users can view other groups and projects as well as their user settings without SSO sign in by using their GitLab.com credentials. + +A user has a SAML identity if one or both of the following are true: + +- They have signed in to GitLab by using their GitLab group's single sign-on URL. +- They were provisioned by SCIM. + +Users are not prompted to sign in through SSO on each visit. GitLab checks +whether a user has authenticated through SSO. If the user last signed in more +than 24 hours ago, GitLab prompts the user to sign in again through SSO. + +SSO is enforced as follows: + +| Project/Group visibility | Enforce SSO setting | Member with identity | Member without identity | Non-member or not signed in | +|--------------------------|---------------------|----------------------|-------------------------|-----------------------------| +| Private | Off | Enforced | Not enforced | Not enforced | +| Private | On | Enforced | Enforced | Enforced | +| Public | Off | Enforced | Not enforced | Not enforced | +| Public | On | Enforced | Enforced | Not enforced | + +An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity. + +### SSO-only for web activity enforcement + +When the **Enforce SSO-only authentication for web activity for this group** option is enabled: + +- All members must access GitLab by using their GitLab group's single sign-on URL + to access group resources, regardless of whether they have an existing SAML + identity. +- SSO is enforced when users access groups and projects in the organization's + group hierarchy. Users can view other groups and projects without SSO sign in. +- Users cannot be added as new members manually. +- Users with the Owner role can use the standard sign in process to make + necessary changes to top-level group settings. +- For non-members or users who are not signed in: + - SSO is not enforced when they access public group resources. + - SSO is enforced when they access private group resources. + +SSO enforcement for web activity has the following effects when enabled: + +- For groups, users cannot share a project in the group outside the top-level + group, even if the project is forked. +- Git activity originating from CI/CD jobs do not have the SSO check enforced. +- Credentials that are not tied to regular users (for example, project and group + access tokens, and deploy keys) do not have the SSO check enforced. +- Users must be signed-in through SSO before they can pull images using the + [Dependency Proxy](../../packages/dependency_proxy/index.md). +- When the **Enforce SSO-only authentication for Git and Dependency Proxy + activity for this group** option is enabled, any API endpoint that involves + Git activity is under SSO enforcement. For example, creating or deleting a + branch, commit, or tag. For Git activity over SSH and HTTPS, users must + have at least one active session signed-in through SSO before they can push to or + pull from a GitLab repository. + +When SSO for web activity is enforced, non-SSO group members do not lose access +immediately. If the user: + +- Has an active session, they can continue accessing the group for up to 24 + hours until the identity provider session times out. +- Is signed out, they cannot access the group after being removed from the + identity provider. -## Passwords for users created via SAML SSO for Groups +## Related topics -The [Generated passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via SAML SSO for Groups. +- [SAML SSO for self-managed GitLab instances](../../../integration/saml.md) +- [Glossary](../../../integration/saml.md#glossary) +- [Authentication comparison between SaaS and self-managed](../../../administration/auth/index.md#saas-vs-self-managed-comparison) +- [Passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) +- [SAML Group Sync](group_sync.md) ## Troubleshooting -See our [troubleshooting SAML guide](troubleshooting.md). +If you find it difficult to match the different SAML terms between GitLab and the +identity provider: + +1. Check your identity provider's documentation. Look at their example SAML + configurations for information on the terms they use. +1. Check the [SAML SSO for self-managed GitLab instances documentation](../../../integration/saml.md). + The self-managed GitLab instance SAML configuration file supports more options + than the GitLab.com file. You can find information on the self-managed instance + file in the: + - External [OmniAuth SAML documentation](https://github.com/omniauth/omniauth-saml/). + - [`ruby-saml` library](https://github.com/onelogin/ruby-saml). +1. Compare the XML response from your provider with our + [example XML used for internal testing](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/fixtures/saml/response.xml). + +For other troubleshooting information, see the [troubleshooting SAML guide](troubleshooting.md). diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index c6ff5dc63c3..38a1c4125aa 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -116,7 +116,7 @@ For each attribute: 1. Select the required settings. 1. Select **Ok**. -If your SAML configuration differs from [the recommended SAML settings](index.md#set-up-azure), select the mapping +If your SAML configuration differs from [the recommended SAML settings](index.md#azure), select the mapping attributes and modify them accordingly. In particular, the `objectId` source attribute must map to the `externalId` target attribute. @@ -133,13 +133,13 @@ Prerequisites: product tier is required to use SCIM on Okta. - [GitLab is configured](#configure-gitlab). - SAML application for [Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/) set up as - described in the [Okta setup notes](index.md#set-up-okta). + described in the [Okta setup notes](index.md#okta). - Your Okta SAML setup matches the [configuration steps exactly](index.md), especially the NameID configuration. To configure Okta for SCIM: 1. Sign in to Okta. -1. Ensure you are in the Admin Area by selecting the **Admin** button located in the upper right. The button is not visible from the Admin Area. +1. In the upper-right corner, select **Admin**. The button is not visible from the Admin Area. 1. In the **Application** tab, select **Browse App Catalog**. 1. Search for **GitLab**, find and select the **GitLab** application. 1. On the GitLab application overview page, select **Add**. @@ -200,6 +200,10 @@ On subsequent visits, new and existing users can access groups either: For role information, see the [Group SAML](index.md#user-access-and-management) page. +### Passwords for users created through SCIM for GitLab groups + +GitLab requires passwords for all user accounts. For more information on how GitLab generates passwords for users created through SCIM for GitLab groups, see [generated passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md). + ### Link SCIM and SAML identities If [group SAML](index.md) is configured and you have an existing GitLab.com account, users can link their SCIM and SAML @@ -210,7 +214,7 @@ To link your SCIM and SAML identities: 1. Update the [primary email](../../profile/index.md#change-your-primary-email) address in your GitLab.com user account to match the user profile email address in your identity provider. -1. [Link your SAML identity](index.md#linking-saml-to-your-existing-gitlabcom-account). +1. [Link your SAML identity](index.md#link-saml-to-your-existing-gitlabcom-account). ### Remove access @@ -222,6 +226,8 @@ Remove or deactivate a user on the identity provider to remove their access to: After the identity provider performs a sync based on its configured schedule, the user's membership is revoked and they lose access. +When you enable SCIM, this does not automatically remove existing users who do not have a SAML identity. + NOTE: Deprovisioning does not delete the GitLab user account. diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md index eadf385feb3..62c13e8909b 100644 --- a/doc/user/group/saml_sso/troubleshooting.md +++ b/doc/user/group/saml_sso/troubleshooting.md @@ -167,7 +167,7 @@ you must set `attribute_statements` in the SAML configuration to This error suggests you are signed in as a GitLab user but have already linked your SAML identity to a different GitLab user. Sign out and then try to sign in again using SAML, which should log you into GitLab with the linked user account. -If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the SAML app](index.md#unlinking-accounts). +If you do not wish to use that GitLab user with the SAML login, you can [unlink the GitLab account from the SAML app](index.md#unlink-accounts). ### Message: "SAML authentication failed: User has already been taken" @@ -176,7 +176,7 @@ Here are possible causes and solutions: | Cause | Solution | | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlinking-accounts) from this GitLab account before attempting to sign in again. | +| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlink-accounts) from this GitLab account before attempting to sign in again. | | The `NameID` changes every time the user requests SSO identification | [Check the `NameID`](#verify-nameid) is not set with `Transient` format, or the `NameID` is not changing on subsequent requests.| ### Message: "SAML authentication failed: Email has already been taken" @@ -196,7 +196,7 @@ User accounts are created in one of the following ways: Getting both of these errors at the same time suggests the `NameID` capitalization provided by the identity provider didn't exactly match the previous value for that user. -This can be prevented by configuring the `NameID` to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlinking-accounts). +This can be prevented by configuring the `NameID` to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlink-accounts). ### Message: "Request to link SAML account must be authorized" @@ -209,7 +209,7 @@ initiated by the service provider and not only the identity provider. ### Message: "There is already a GitLab account associated with this email address. Sign in with your existing credentials to connect your organization's account" **(PREMIUM SAAS)** -A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account). +A user can see this message when they are trying to [manually link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account). To resolve this problem, the user should check they are using the correct GitLab password to sign in. The user first needs to [reset their password](https://gitlab.com/users/password/new) if both: @@ -233,7 +233,7 @@ This can then be compared to the `NameID` being sent by the identity provider by Ensure that the **GitLab single sign-on URL** (for GitLab.com) or the instance URL (for self-managed) has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. -For GitLab.com, alternatively, when users need to [link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. +For GitLab.com, alternatively, when users need to [link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. ### Users receive a 404 **(PREMIUM SAAS)** @@ -244,8 +244,8 @@ If all users are receiving a `404` when attempting to sign in using SAML, confir If you receive a `404` during setup when using "verify configuration", make sure you have used the correct [SHA-1 generated fingerprint](../../../integration/saml.md#configure-saml-on-your-idp). -If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#configure-your-identity-provider), they may see a 404. -As outlined in the [user access section](index.md#linking-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. +If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](index.md#set-up-your-identity-provider), they may see a 404. +As outlined in the [user access section](index.md#link-saml-to-your-existing-gitlabcom-account), a group Owner needs to provide the URL to users. If all users are receiving a `404` after signing in to the identity provider (IdP): @@ -317,4 +317,4 @@ This error appears when you try to invite a user to a GitLab.com group (or subgr If you see this message after trying to invite a user to a group: 1. Ensure the user has been [added to the SAML identity provider](index.md#user-access-and-management). -1. Ask the user to [link SAML to their existing GitLab.com account](index.md#linking-saml-to-your-existing-gitlabcom-account), if they have one. Otherwise, ask the user to create a GitLab.com account by [accessing GitLab.com through the identity provider's dashboard](index.md#user-access-and-management), or by [signing up manually](https://gitlab.com/users/sign_up) and linking SAML to their new account. +1. Ask the user to [link SAML to their existing GitLab.com account](index.md#link-saml-to-your-existing-gitlabcom-account), if they have one. Otherwise, ask the user to create a GitLab.com account by [accessing GitLab.com through the identity provider's dashboard](index.md#user-access-and-management), or by [signing up manually](https://gitlab.com/users/sign_up) and linking SAML to their new account. diff --git a/doc/user/group/saml_sso/troubleshooting_scim.md b/doc/user/group/saml_sso/troubleshooting_scim.md index 939ed804a99..33343c9b339 100644 --- a/doc/user/group/saml_sso/troubleshooting_scim.md +++ b/doc/user/group/saml_sso/troubleshooting_scim.md @@ -79,7 +79,7 @@ You must not: When the SCIM app changes: -- Users can follow the instructions in the [Change the SAML app](index.md#change-the-saml-app) section. +- Users can follow the instructions in the [Change the SAML app](index.md#change-the-identity-provider) section. - Administrators of the identity provider can: 1. Remove users from the SCIM app, which unlinks all removed users. 1. Turn on sync for the new SCIM app to [link existing users](scim_setup.md#link-scim-and-saml-identities). @@ -163,3 +163,27 @@ error. The error response can include a HTML result of the GitLab URL `https://g This error is harmless and occurs because group provisioning was turned on but GitLab SCIM integration does not support it nor require it. To remove the error, follow the instructions in the Azure configuration guide to disable the option to [synchronize Azure Active Directory groups to AppName](scim_setup.md#configure-azure-active-directory). + +## Okta + +The following troubleshooting information is specifically for SCIM provisioned through Okta. + +### `Error authenticating: null` message when testing API SCIM credentials + +When testing the API credentials in your Okta SCIM application, you may encounter an error: + +```plaintext +Error authenticating: null +``` + +Okta needs to be able to connect to your GitLab instance to provision or deprovision users. + +In your Okta SCIM application, check that the SCIM **Base URL** is correct and pointing to a valid GitLab +SCIM API endpoint URL. Check the following documentation to find information on this URL for: + +- [GitLab.com groups](scim_setup.md#configure-gitlab). +- [Self-managed GitLab instances](../../admin_area/settings/scim_setup.md#configure-gitlab). + +For self-managed GitLab instances, ensure that GitLab is publicly available so Okta can connect to it. If needed, +you can [allow access to Okta IP addresses](https://help.okta.com/en-us/Content/Topics/Security/ip-address-allow-listing.htm) +on your firewall. diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index cd50c209b0d..4a2bfd4c4b4 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -28,16 +28,12 @@ associated with a group rather than a project or user. In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. WARNING: -The ability to create group access tokens without expiry was -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab -16.0. When this ability is removed, existing group access tokens without an expiry are planned to have an expiry added. -The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry -occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. +The ability to create group access tokens without an expiry date was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. In GitLab 16.0 and later, existing group access tokens without an expiry date are automatically given an expiry date 365 days later than the current date. The automatic adding of an expiry date occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry date occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. You can use group access tokens: -- On GitLab SaaS if you have the Premium license tier or higher. Group access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). -- On self-managed instances of GitLab, with any license tier. If you have the Free tier: +- On GitLab SaaS: If you have the Premium or Ultimate license tier. Group access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). +- On self-managed instances: With any license tier. If you have the Free tier: - Review your security and compliance policies around [user self-enrollment](../../admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups). - Consider [disabling group access tokens](#enable-or-disable-group-access-token-creation) to @@ -48,31 +44,33 @@ You cannot use group access tokens to create other group, project, or personal a Group access tokens inherit the [default prefix setting](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) configured for personal access tokens. -NOTE: -Group access tokens are not FIPS compliant and creation and use are disabled when [FIPS mode](../../../development/fips_compliance.md) is enabled. - ## Create a group access token using UI > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214045) in GitLab 14.7. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days and default role of Guest is populated in the UI. +> - Ability to create non-expiring group access tokens [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. + +WARNING: +Project access tokens are treated as [internal users](../../../development/internal_users.md). +If an internal user creates a project access token, that token is able to access +all projects that have visibility level set to [Internal](../../public_access.md). To create a group access token: 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the group. -1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. +1. Enter an expiry date for the token: + - The token expires on that date at midnight UTC. + - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. + - By default, this date can be a maximum of 365 days later than the current date. + - An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-group-access-token). 1. Select **Create group access token**. A group access token is displayed. Save the group access token somewhere safe. After you leave or refresh the page, you can't view it again. -WARNING: -Group access tokens are treated as [internal users](../../../development/internal_users.md). -If an internal user creates a group access token, that token is able to access all -groups that have visibility level set to [Internal](../../public_access.md). - ## Create a group access token using Rails console GitLab 14.6 and earlier doesn't support creating group access tokens using the UI @@ -87,8 +85,9 @@ or API. However, administrators can use a workaround: # Set the group group you want to create a token for. For example, group with ID 109. group = Group.find(109) - # Create the group bot user. For further group access tokens, the username should be group_#{group.id}_bot#{bot_count}. For example, group_109_bot2 and email address group_109_bot2@example.com. - bot = Users::CreateService.new(admin, { name: 'group_token', username: "group_#{group.id}_bot", email: "group_#{group.id}_bot@example.com", user_type: :project_bot }).execute + # Create the group bot user. For further group access tokens, the username should be `group_{group_id}_bot_{random_string}` and email address `group_{group_id}_bot_{random_string}@noreply.{Gitlab.config.gitlab.host}`. + random_string = SecureRandom.hex(16) + bot = Users::CreateService.new(admin, { name: 'group_token', username: "group_#{group.id}_bot_#{random_string}", email: "group_#{group.id}_bot_#{random_string}@noreply.#{Gitlab.config.gitlab.host}", user_type: :project_bot }).execute # Confirm the group bot. bot.confirm @@ -172,7 +171,11 @@ to groups instead of projects. Bot users for groups: - Do not count as licensed seats. - Can have a maximum role of Owner for a group. For more information, see [Create a group access token](../../../api/group_access_tokens.md#create-a-group-access-token). -- Have a username set to `group_{group_id}_bot` for the first access token. For example, `group_123_bot`. -- Have an email set to `group{group_id}_bot@noreply.{Gitlab.config.gitlab.host}`. For example, `group123_bot@noreply.example.com`. +- Have a username set to `group_{group_id}_bot_{random_string}`. For example, `group_123_bot_4ffca233d8298ea1`. +- Have an email set to `group_{group_id}_bot_{random_string}@noreply.{Gitlab.config.gitlab.host}`. For example, `group_123_bot_4ffca233d8298ea1@noreply.example.com`. All other properties are similar to [bot users for projects](../../project/settings/project_access_tokens.md#bot-users-for-projects). + +## Token availability + +Group access tokens are only available in paid subscriptions, and not available in trial subscriptions. For more information, see the ["What is included" section of the GitLab Trial FAQ](https://about.gitlab.com/free-trial/#what-is-included-in-my-free-trial-what-is-excluded). diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md deleted file mode 100644 index ff64a7dcd54..00000000000 --- a/doc/user/group/settings/import_export.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../import/index.md' -remove_date: '2023-03-08' ---- - -This document was moved to [another location](../import/index.md). - -<!-- This redirect file can be deleted after <2023-03-08>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 6c7baa848e7..63ffbf62981 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -85,7 +85,7 @@ You cannot host a GitLab Pages subgroup website with a top-level domain name. Fo To create a subgroup: 1. On the top bar, select **Main menu > Groups** and find and select the parent group to add a subgroup to. -1. On the parent group's overview page, in the upper right, select **New subgroup**. +1. On the parent group's overview page, in the upper-right corner, select **New subgroup**. 1. Select **Create group**. 1. Fill in the fields. View a list of [reserved names](../../reserved_names.md) that cannot be used as group names. 1. Select **Create group**. diff --git a/doc/user/group/value_stream_analytics/img/object_hierarchy_example_V14_10.png b/doc/user/group/value_stream_analytics/img/object_hierarchy_example_V14_10.png Binary files differnew file mode 100644 index 00000000000..9c4d4ae7718 --- /dev/null +++ b/doc/user/group/value_stream_analytics/img/object_hierarchy_example_V14_10.png diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index c5a95779087..bc48c1050fb 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -5,9 +5,9 @@ group: Optimize info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Value stream analytics for groups **(PREMIUM)** +# Value stream analytics **(FREE)** -Value stream analytics provides metrics about each stage of your software development process. +Value stream analytics measures the time it takes to go from an idea to production. A **value stream** is the entire work process that delivers value to customers. For example, the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts @@ -21,111 +21,83 @@ Use value stream analytics to identify: - Long-running issues or merge requests. - Factors that cause your software development lifecycle to slow down. -Value stream analytics is also available for [projects](../../analytics/value_stream_analytics.md). +Value stream analytics helps businesses: -## View value stream analytics +- Visualize their end-to-end DevSecOps workstreams. +- Identify and solve inefficiencies. +- Optimize their workstreams to deliver more value, faster. -> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 13.3 -> - Horizontal stage path [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in 13.0 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in 13.12 +Value stream analytics is available for projects and groups. -Prerequisites: +## Feature availability -- You must have at least the Reporter role to view value stream analytics for groups. -- You must create a [custom value stream](#create-a-value-stream-with-gitlab-default-stages). Value stream analytics only shows custom value streams created for your group. +Value stream analytics offers different features at the project and group level for FOSS and licensed versions. -To view value stream analytics for your group: +- On GitLab Free, value stream analytics does not aggregate data. It queries the database directly where the date range filter is applied to the creation date of issues and merge request. You can view value stream analytics with pre-defined default stages. +- On GitLab Premium, value stream analytics aggregates data and applies the date range filter on the end event. You can also create, edit, and delete value streams. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. To view metrics for a particular stage, select a stage below the **Filter results** text box. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. The charts and list show workflow items created - during the date range. -1. Optional. Sort results by ascending or descending: - - To sort by most recent or oldest workflow item, select the **Last event** header. - - To sort by most or least amount of time spent in each stage, select the **Duration** header. +|Feature|Group level (licensed)|Project level (licensed)|Project level (FOSS)| +|-|-|-|-| +|Create custom value streams|Yes|Yes|no, only one value stream (default) is present with the default stages| +|Create custom stages|Yes|Yes|No| +|Filtering (for example, by author, label, milestone)|Yes|Yes|Yes| +|Stage time chart|Yes|Yes|No| +|Total time chart|Yes|Yes|No| +|Task by type chart|Yes|No|No| +|DORA Metrics|Yes|Yes|No| +|Cycle time and lead time summary (Key metrics)|Yes|Yes|No| +|New issues, commits, and deploys (Key metrics)|Yes, excluding commits|Yes|Yes| +|Uses aggregated backend|Yes|Yes|No| +|Date filter behavior|Filters items [finished within the date range](https://gitlab.com/groups/gitlab-org/-/epics/6046)|Filters items by creation date.|Filters items by creation date.| +|Authorization|At least reporter|At least reporter|Can be public| -A badge next to the workflow items table header shows the number of workflow items that -completed during the selected stage. +## How value stream analytics works -The table shows a list of related workflow items for the selected stage. Based on the stage you select, this can be: +Value stream analytics calculates the duration of every stage of your software development process. -- CI/CD jobs -- Issues -- Merge requests -- Pipelines +Value stream analytics is made of three core objects: -## View DORA metrics and key metrics for a group +- A **value stream** contains a value stream stage list. +- Each value stream stage list contains one or more **stages**. +- Each stage has two **events**: start and stop. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. +### Value stream stages -The **Overview** dashboard in value stream analytics shows key metrics and DORA metrics of group performance. Based on the filter you select, -the dashboard automatically aggregates DORA metrics and displays the current status of the value stream. Select a DORA metric to view its chart. +A stage represents an event pair (start and end events) with additional metadata, such as the name of the stage. You can configure the stages in the pairing rules defined in the backend. -Prerequisite: +### Value streams -- To view deployment metrics, you must have a -[production environment configured](../../../ci/environments/index.md#deployment-tier-of-environments). +Value streams are container objects for the stages. You can have multiple value streams per group, to focus on different aspects of the DevOps lifecycle. -To view the DORA metrics and key metrics: +### Value stream stage events -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -Key metrics and DORA metrics display below the **Filter results** text box. - -### Key metrics in the value stream - -The **Overview** dashboard shows the following key metrics that measure team performance: - -- Lead time: Median time from when the issue was created to when it was closed. -- Cycle time: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a - [linked issue's merge request](../../project/issues/crosslinking_issues.md#from-commit-messages) to when that issue is closed. - The cycle time approach underestimates the lead time because merge request creation is always later than commit time. -- New issues: Number of new issues created. -- Deploys: Total number of deployments to production. - -### DORA metrics **(ULTIMATE)** +Events are the smallest building blocks of the value stream analytics feature. A stage consists of a start event and an end event. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) lead time for changes DORA metric in GitLab 14.5. -> - DORA API-based deployment metrics for value stream analytics for groups were [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/337256) from GitLab Ultimate to GitLab Premium in GitLab 14.3. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355304) time to restore service tile in GitLab 15.0. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357071) change failure rate tile in GitLab 15.0. +The following stage events are available: -The value stream analytics **Overview** dashboard displays the following [DORA](../../../user/analytics/dora_metrics.md) metrics: - -- Deployment Frequency. -- Lead time for changes. -- Time to restore service. -- Change failure rate. - -DORA metrics are calculated based on data from the -[DORA API](../../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). +- Issue closed +- Issue created +- Issue first added to board +- Issue first assigned +- Issue first associated with milestone +- Issue first mentioned +- Issue label added +- Issue label removed +- MR closed +- MR merged +- MR created +- MR first commit time +- MR first assigned +- MR first deployed +- MR label added +- MR label removed +- MR last pipeline duration -NOTE: -In GitLab 13.9 and later, deployment frequency metrics are calculated based on when the deployment was finished. -In GitLab 13.8 and earlier, deployment frequency metrics are calculated based on when the deployment was created. +These events play a key role in the duration calculation, which is calculated by the formula: duration = end event time - start event time. -<div class="video-fallback"> - See the video: <a href="https://www.youtube.com/watch?v=wQU-mWvNSiI">DORA metrics and value stream analytics</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/wQU-mWvNSiI" frameborder="0" allowfullscreen> </iframe> -</figure> +To learn what start and end events can be paired, see [Validating start and end events](../../../development/value_stream_analytics.md#validating-start-and-end-events). -### How value stream analytics aggregates data +### How value stream analytics aggregates data **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.5. > - Filter by stop date toggle [added](https://gitlab.com/gitlab-org/gitlab/-/issues/352428) in GitLab 14.9 @@ -147,31 +119,7 @@ longer than 10 minutes in the following cases: To view when the data was most recently updated, in the right corner next to **Edit**, hover over the **Last updated** badge. -## View metrics for each development stage - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. - -Value stream analytics shows the median time spent by issues or merge requests in each development stage. - -To view the median time spent in each stage by a group: - -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. Optional. Filter the results: - 1. Select the **Filter results** text box. - 1. Select a parameter. - 1. Select a value or enter text to refine the results. - 1. To adjust the date range: - - In the **From** field, select a start date. - - In the **To** field, select an end date. -1. To view the metrics for each stage, above the **Filter results** text box, hover over a stage. - -NOTE: -The date range selector filters items by the event time. The event time is when the -selected stage finished for the given item. - -## How value stream analytics measures stages +### How value stream analytics measures stages Value stream analytics measures each stage from its start event to its end event. @@ -194,7 +142,7 @@ Each pre-defined stages of value stream analytics is further described in the ta For information about how value stream analytics calculates each stage, see the [Value stream analytics development guide](../../../development/value_stream_analytics.md). -### Example workflow +#### Example workflow This example shows a workflow through all seven stages in one day. @@ -234,9 +182,9 @@ Keep in mind the following observations related to this example: - This example illustrates only **one cycle** of the seven stages. The value stream analytics dashboard shows the median time for multiple cycles. -## How value stream analytics identifies the production environment +### How value stream analytics identifies the production environment -Value stream analytics identifies production environments by looking for project +Value stream analytics identifies [production environments](../../../ci/environments/index.md#deployment-tier-of-environments) by looking for project [environments](../../../ci/yaml/index.md#environment) with a name matching any of these patterns: - `prod` or `prod/*` @@ -246,14 +194,189 @@ These patterns are not case-sensitive. You can change the name of a project environment in your GitLab CI/CD configuration. -## Create a value stream with GitLab default stages +## View value stream analytics + +> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 13.3 +> - Horizontal stage path [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in 13.0 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in 13.12 + +Prerequisites: + +- You must have at least the Reporter role. +- You must create a [custom value stream](#create-a-value-stream-with-gitlab-default-stages). Value stream analytics only shows custom value streams created for your group or project. + +To view value stream analytics for your group or project: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. To view metrics for a particular stage, select a stage below the **Filter results** text box. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. The charts and list show workflow items created + during the date range. +1. Optional. Sort results by ascending or descending: + - To sort by most recent or oldest workflow item, select the **Last event** header. + - To sort by most or least amount of time spent in each stage, select the **Duration** header. + +A badge next to the workflow items table header shows the number of workflow items that +completed during the selected stage. + +The table shows a list of related workflow items for the selected stage. Based on the stage you select, this can be: + +- Issues +- Merge requests + +### Data filters + +You can filter value stream analytics to view data that matches specific criteria. The following filters are supported: + +- Date range +- Project +- Assignee +- Author +- Milestone +- Label + +NOTE: +For the "Tasks by type" chart, only the Date range and Project selector filters are available. Labels and other filters are not applied, and you need to select labels separately from the dropdown list next to the chart. + +## Value stream analytics metrics + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. + +The **Overview** page in value stream analytics displays key metrics of the DevSecOps lifecycle performance for projects and groups. + +### Key metrics + +Value stream analytics includes the following key metrics: + +- **Lead time**: Median time from when the issue was created to when it was closed. +- **Cycle time**: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a + [linked issue's merge request](../../project/issues/crosslinking_issues.md#from-commit-messages) to when that issue is closed. + The cycle time approach underestimates the lead time because merge request creation is always later than commit time. +- **New issues**: Number of new issues created. +- **Deploys**: Total number of deployments to production. + +### DORA metrics **(ULTIMATE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) lead time for changes DORA metric in GitLab 14.5. +> - DORA API-based deployment metrics for value stream analytics for groups were [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/337256) from GitLab Ultimate to GitLab Premium in GitLab 14.3. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355304) time to restore service tile in GitLab 15.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357071) change failure rate tile in GitLab 15.0. + +Value stream analytics includes the following [DORA](../../../user/analytics/dora_metrics.md) metrics: + +- Deployment frequency +- Lead time for changes +- Time to restore service +- Change failure rate + +DORA metrics are calculated based on data from the +[DORA API](../../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). + +If you have a GitLab Premium or Ultimate subscription: + +- The number of successful deployments is calculated with DORA data. +- The data is filtered based on environment and environment tier. + +NOTE: +In GitLab 13.9 and later, deployment frequency metrics are calculated based on when the deployment was finished. +In GitLab 13.8 and earlier, deployment frequency metrics are calculated based on when the deployment was created. + +## View key and DORA metrics + +Prerequisite: + +- To view deployment metrics, you must have a +[production environment configured](#how-value-stream-analytics-identifies-the-production-environment). + +To view key lifecycle metrics: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. + Key metrics display below the **Filter results** text box. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + Based on the filter you select, the dashboard automatically aggregates key metrics and displays the status of the value stream. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. + +To view the [Value Streams Dashboard](../../analytics/value_streams_dashboard.md) and [DORA metrics](../../analytics/dora_metrics.md): + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, in the **Key metrics** row, select **Value Streams Dashboard / DORA**. +1. Optional. To open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL + (for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). + +## View metrics for each development stage + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. + +Value stream analytics shows the median time spent by issues or merge requests in each development stage. + +To view the median time spent in each stage by a group: + +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. +1. To view the metrics for each stage, above the **Filter results** text box, hover over a stage. + +NOTE: +The date range selector filters items by the event time. The event time is when the +selected stage finished for the given item. + +## View tasks by type **(PREMIUM)** + +The **Tasks by type** chart displays the cumulative number of issues and merge requests per day for your group. + +The chart uses the global page filters to display data based on the selected +group and time frame. + +To view tasks by type: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Below the **Filter results** text box, select **Overview**. The **Tasks by type** chart displays below the **Total time** chart. +1. To switch between the task type, select the **Settings** (**{settings}**) dropdown list + and select **Issues** or **Merge Requests**. +1. To add or remove labels, select the **Settings** (**{settings}**) dropdown list + and select or search for a label. By default the top group-level labels (maximum 10) are selected. You can select a maximum of 15 labels. + +## Create a value stream **(PREMIUM)** + +### Create a value stream with GitLab default stages > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221202) in GitLab 13.3 When you create a value stream, you can use GitLab default stages and hide or re-order them. You can also create custom stages in addition to those provided in the default template. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create new Value Stream**. 1. Enter a name for the value stream. @@ -269,7 +392,7 @@ create custom stages in addition to those provided in the default template. NOTE: If you have recently upgraded to GitLab Premium, it can take up to 30 minutes for data to collect and display. -## Create a value stream with custom stages +### Create a value stream with custom stages > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50229) in GitLab 13.7. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55572) in GitLab 13.10. @@ -277,7 +400,9 @@ If you have recently upgraded to GitLab Premium, it can take up to 30 minutes fo When you create a value stream, you can create and add custom stages that align with your own development workflows. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. Select **Create value stream**. 1. For each stage: @@ -287,7 +412,7 @@ When you create a value stream, you can create and add custom stages that align 1. To re-order the stages, select the up or down arrows. 1. Select **Create value stream**. -### Label-based stages for custom value streams +#### Label-based stages for custom value streams To measure complex workflows, you can use [scoped labels](../../project/labels.md#scoped-labels). For example, to measure deployment time from a staging environment to production, you could use the following labels: @@ -297,15 +422,25 @@ time from a staging environment to production, you could use the following label ![Label-based value stream analytics stage](img/vsa_label_based_stage_v14_0.png "Creating a label-based value stream analytics stage") -## Edit a value stream +#### Example for custom value stream configuration + +![Example configuration](img/object_hierarchy_example_V14_10.png "Example custom value stream configuration") + +In the example above, two independent value streams are set up for two teams that are using different development workflows in the **Test Group** (top-level namespace). + +The first value stream uses standard timestamp-based events for defining the stages. The second value stream uses label events. + +## Edit a value stream **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267537) in GitLab 13.10. After you create a value stream, you can customize it to suit your purposes. To edit a value stream: -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. -1. In the upper right, select the dropdown list, and select a value stream. +1. In the upper-right corner, select the dropdown list, then select a value stream. 1. Next to the value stream dropdown list, select **Edit**. 1. Optional: - Rename the value stream. @@ -316,21 +451,22 @@ After you create a value stream, you can customize it to suit your purposes. To 1. Optional. To undo any modifications, select **Restore value stream defaults**. 1. Select **Save Value Stream**. -## Delete a value stream +## Delete a value stream **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221205) in GitLab 13.4. To delete a custom value stream: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value Stream**. -1. In the upper right, select the dropdown list and then select the value stream you would like to delete. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. In the upper-right corner, select the dropdown list, then select the value stream you would like to delete. 1. Select **Delete (name of value stream)**. 1. To confirm, select **Delete**. ![Delete value stream](img/delete_value_stream_v13_12.png "Deleting a custom value stream") -## View number of days for a cycle to complete +## View number of days for a cycle to complete **(PREMIUM)** > - Chart median line [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235455) in GitLab 13.4. > - Totals [replaced](https://gitlab.com/gitlab-org/gitlab/-/issues/262070) with averages in GitLab 13.12. @@ -338,7 +474,9 @@ To delete a custom value stream: The **Total time chart** shows the average number of days it takes for development cycles to complete. The chart shows data for the last 500 workflow items. -1. On the top bar, select **Main menu > Groups** and find your group. +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. 1. Above the **Filter results** box, select a stage: - To view a summary of the cycle time for all stages, select **Overview**. @@ -351,23 +489,42 @@ The chart shows data for the last 500 workflow items. - In the **From** field, select a start date. - In the **To** field, select an end date. -## View tasks by type +## Access permissions for value stream analytics -The **Tasks by type** chart displays the cumulative number of issues and merge requests per day for your group. +Access permissions for value stream analytics depend on the project type. -The chart uses the global page filters to display data based on the selected -group, projects, and time frame. +| Project type | Permissions | +|--------------|----------------------------------------| +| Public | Anyone can access. | +| Internal | Any authenticated user can access. | +| Private | Any member Guest and above can access. | -To view tasks by type: +## Troubleshooting -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Analytics > Value stream**. -1. Below the **Filter results** text box, select **Overview**. The **Tasks by type** chart displays below the **Total time** chart. -1. To switch between the task type, select the **Settings** (**{settings}**) dropdown list - and select **Issues** or **Merge Requests**. -1. To add or remove labels, select the **Settings** (**{settings}**) dropdown list - and select or search for a label. By default the top group-level labels (maximum 10) are selected. You can select a maximum of 15 labels. +### 100% CPU utilization by Sidekiq `cronjob:analytics_cycle_analytics` -## Troubleshooting +It is possible that value stream analytics background jobs +strongly impact performance by monopolizing CPU resources. + +To recover from this situation: + +1. Disable the feature for all projects in [the Rails console](../../../administration/operations/rails_console.md), + and remove existing jobs: + + ```ruby + Project.find_each do |p| + p.analytics_access_level='disabled'; + p.save! + end + + Analytics::CycleAnalytics::GroupStage.delete_all + Analytics::CycleAnalytics::Aggregation.delete_all + ``` -See [Value stream analytics for projects](../../analytics/value_stream_analytics.md#troubleshooting). +1. Configure a [Sidekiq routing](../../../administration/sidekiq/processing_specific_job_classes.md) + with for example a single `feature_category=value_stream_management` + and multiple `feature_category!=value_stream_management` entries. + Find other relevant queue metadata in the + [Enterprise Edition list](../../../administration/sidekiq/processing_specific_job_classes.md#list-of-available-job-classes). +1. Enable value stream analytics for one project after another. + You might need to tweak the Sidekiq routing further according to your performance requirements. diff --git a/doc/user/img/explain_code_experiment.png b/doc/user/img/explain_code_experiment.png Binary files differnew file mode 100644 index 00000000000..1b3b9e3eef3 --- /dev/null +++ b/doc/user/img/explain_code_experiment.png diff --git a/doc/user/img/explain_this_vulnerability.png b/doc/user/img/explain_this_vulnerability.png Binary files differnew file mode 100644 index 00000000000..0880ad5f875 --- /dev/null +++ b/doc/user/img/explain_this_vulnerability.png diff --git a/doc/user/img/get_domain_verification_code_v16_0.png b/doc/user/img/get_domain_verification_code_v16_0.png Binary files differnew file mode 100644 index 00000000000..34c77866865 --- /dev/null +++ b/doc/user/img/get_domain_verification_code_v16_0.png diff --git a/doc/user/img/retry_domain_verification_v16_0.png b/doc/user/img/retry_domain_verification_v16_0.png Binary files differnew file mode 100644 index 00000000000..4183fd5a342 --- /dev/null +++ b/doc/user/img/retry_domain_verification_v16_0.png diff --git a/doc/user/img/todos_add_okrs_v16_0.png b/doc/user/img/todos_add_okrs_v16_0.png Binary files differnew file mode 100644 index 00000000000..45c04e647e2 --- /dev/null +++ b/doc/user/img/todos_add_okrs_v16_0.png diff --git a/doc/user/img/todos_mark_done_okrs_v16_0.png b/doc/user/img/todos_mark_done_okrs_v16_0.png Binary files differnew file mode 100644 index 00000000000..545b3569ed5 --- /dev/null +++ b/doc/user/img/todos_mark_done_okrs_v16_0.png diff --git a/doc/user/infrastructure/clusters/connect/index.md b/doc/user/infrastructure/clusters/connect/index.md index 9325f11b0c7..b571a65b50a 100644 --- a/doc/user/infrastructure/clusters/connect/index.md +++ b/doc/user/infrastructure/clusters/connect/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -10,7 +10,7 @@ The [certificate-based Kubernetes integration with GitLab](../index.md) was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your clusters, use the [GitLab agent](../../../clusters/agent/index.md). -## Cluster levels (DEPRECATED) +## Cluster levels (deprecated) > [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md index df785cce406..45445c0a0f5 100644 --- a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -117,18 +117,18 @@ To remove all resources: 1. Add the following to your `.gitlab-ci.yml` file: - ```yaml - stages: - - init - - validate - - build - - deploy - - cleanup - - destroy: - extends: .destroy - needs: [] - ``` + ```yaml + stages: + - init + - validate + - build + - deploy + - cleanup + + destroy: + extends: .destroy + needs: [] + ``` 1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md index cefa8885bfe..4516ea538a9 100644 --- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -67,42 +67,42 @@ Set up your AWS credentials when you want to authenticate AWS with GitLab. 1. Create an [IAM User](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html) or [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html). 1. Make sure that your IAM user or role has the appropriate permissions for your project. For this example project, you must have the permissions shown below. You can expand this when you set up your own project. - ```json - // IAM custom Policy definition - { - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "VisualEditor0", - "Effect": "Allow", - "Action": [ - "ec2:*", - "eks:*", - "elasticloadbalancing:*", - "autoscaling:*", - "cloudwatch:*", - "logs:*", - "kms:DescribeKey", - "iam:AddRoleToInstanceProfile", - "iam:AttachRolePolicy", - "iam:CreateInstanceProfile", - "iam:CreateRole", - "iam:CreateServiceLinkedRole", - "iam:GetRole", - "iam:ListAttachedRolePolicies", - "iam:ListRolePolicies", - "iam:ListRoles", - "iam:PassRole", - // required for destroy step - "iam:DetachRolePolicy", - "iam:ListInstanceProfilesForRole", - "iam:DeleteRole" - ], - "Resource": "*" - } - ] - } - ``` + ```json + // IAM custom Policy definition + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": [ + "ec2:*", + "eks:*", + "elasticloadbalancing:*", + "autoscaling:*", + "cloudwatch:*", + "logs:*", + "kms:DescribeKey", + "iam:AddRoleToInstanceProfile", + "iam:AttachRolePolicy", + "iam:CreateInstanceProfile", + "iam:CreateRole", + "iam:CreateServiceLinkedRole", + "iam:GetRole", + "iam:ListAttachedRolePolicies", + "iam:ListRolePolicies", + "iam:ListRoles", + "iam:PassRole", + // required for destroy step + "iam:DetachRolePolicy", + "iam:ListInstanceProfilesForRole", + "iam:DeleteRole" + ], + "Resource": "*" + } + ] + } + ``` 1. [Create an access key for the user or role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). 1. Save your access key and secret. You need these to authenticate AWS with GitLab. @@ -165,19 +165,19 @@ To remove all resources: 1. Add the following to your `.gitlab-ci.yml` file: - ```yaml - stages: - - init - - validate - - test - - build - - deploy - - cleanup - - destroy: - extends: .terraform:destroy - needs: [] - ``` + ```yaml + stages: + - init + - validate + - test + - build + - deploy + - cleanup + + destroy: + extends: .terraform:destroy + needs: [] + ``` 1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index 7e6a0495d90..c8d2fb674b2 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -143,19 +143,19 @@ To remove all resources: 1. Add the following to your `.gitlab-ci.yml` file: - ```yaml - stages: - - init - - validate - - build - - test - - deploy - - cleanup - - destroy: - extends: .terraform:destroy - needs: [] - ``` + ```yaml + stages: + - init + - validate + - build + - test + - deploy + - cleanup + + destroy: + extends: .terraform:destroy + needs: [] + ``` 1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md index 6bd44c7a0f7..50185966267 100644 --- a/doc/user/infrastructure/clusters/deploy/inventory_object.md +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/index.md b/doc/user/infrastructure/clusters/index.md index 14fd4917141..d0419e08f95 100644 --- a/doc/user/infrastructure/clusters/index.md +++ b/doc/user/infrastructure/clusters/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To connect clusters to GitLab, use the [GitLab agent](../../clusters/agent/index.md). -## Certificate-based Kubernetes integration (DEPRECATED) +## Certificate-based Kubernetes integration (deprecated) WARNING: In GitLab 14.5, the certificate-based method to connect Kubernetes clusters diff --git a/doc/user/infrastructure/clusters/manage/clusters_health.md b/doc/user/infrastructure/clusters/manage/clusters_health.md index 4f63f3e4e2a..d9b849ffccc 100644 --- a/doc/user/infrastructure/clusters/manage/clusters_health.md +++ b/doc/user/infrastructure/clusters/manage/clusters_health.md @@ -1,19 +1,12 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../index.md' --- -# Clusters health (DEPRECATED) **(FREE)** +# Clusters health (removed) **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in GitLab 10.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/208224) from GitLab Ultimate to GitLab Free in 13.2. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. However, you can **still use** Prometheus -for Kubernetes clusters connected to GitLab by [enabling Prometheus manually](../../../project/integrations/prometheus.md#manual-configuration-of-prometheus). - -When [the Prometheus cluster integration is enabled](../../../clusters/integrations.md#prometheus-cluster-integration), GitLab monitors the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start. - -![Cluster Monitoring](img/k8s_cluster_monitoring.png) +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/infrastructure/clusters/manage/img/k8s_cluster_monitoring.png b/doc/user/infrastructure/clusters/manage/img/k8s_cluster_monitoring.png Binary files differdeleted file mode 100644 index 0a8c5043c65..00000000000 --- a/doc/user/infrastructure/clusters/manage/img/k8s_cluster_monitoring.png +++ /dev/null diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md index 8b75d54d352..f07b70dd8e0 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md index 14d3a7996e0..1d56e7c1ba1 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md index c2190ad7cfa..bbf8391e8a0 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index b2b5da61a81..a21c7271e7a 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/gitlab_terraform_helpers.md b/doc/user/infrastructure/iac/gitlab_terraform_helpers.md index 8be949c40cd..d5376bfbcb1 100644 --- a/doc/user/infrastructure/iac/gitlab_terraform_helpers.md +++ b/doc/user/infrastructure/iac/gitlab_terraform_helpers.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index f9891934067..d4fc345f494 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -100,7 +100,7 @@ For GitLab-curated template recipes, see [Terraform template recipes](terraform_ ## Related topics - View [the images that contain the `gitlab-terraform` shell script](https://gitlab.com/gitlab-org/terraform-images). -- Use GitLab as a [Terraform module registry](../../packages/terraform_module_registry/index.md). +- Use GitLab as a [Terraform Module Registry](../../packages/terraform_module_registry/index.md). - To store state files in local storage or in a remote store, use the [GitLab-managed Terraform state](terraform_state.md). - To collaborate on Terraform code changes and Infrastructure-as-Code workflows, use the [Terraform integration in merge requests](mr_integration.md). diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index 7b4a7cd6b95..b2e02cd7375 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 93a82023480..1b0065fd165 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -1,16 +1,14 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # GitLab-managed Terraform state **(FREE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. -> - Support for state names that contain periods introduced in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. Disabled by default. [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106861) in GitLab 15.7. - -FLAG: -On self-managed GitLab, by default support for state names that contain periods is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. On GitLab.com, support for state names that contain periods is available. Requests for state files might generate HTTP 404 errors after enabling this feature. For more information, see [Troubleshooting the Terraform integration with GitLab](troubleshooting.md#state-not-found-if-the-state-name-contains-a-period). +> - Support for state names that contain periods introduced in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `allow_dots_on_tf_state_names`. Disabled by default. +> - Support for state names that contain periods [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/385597) in GitLab 16.0. Feature flag `allow_dots_on_tf_state_names` removed. Terraform uses state files to store details about your infrastructure configuration. With Terraform remote [backends](https://www.terraform.io/language/settings/backends/configuration), @@ -42,7 +40,7 @@ For self-managed GitLab, before you can use GitLab for your Terraform state file - An administrator must [set up Terraform state storage](../../../administration/terraform_state.md). - You must enable the **Infrastructure** menu for your project. Go to **Settings > General**, - expand **Visibility, project features, permissions**, and under **Operations**, turn on the toggle. + expand **Visibility, project features, permissions**, and under **Infrastructure**, turn on the toggle. ## Initialize a Terraform state as a backend by using GitLab CI/CD @@ -58,7 +56,7 @@ WARNING: Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan includes sensitive data, like passwords, access tokens, or certificates, you should -encrypt plan output or modify the project visibility settings. We also strongly recommend that you **disable** +encrypt plan output or modify the project visibility settings. We also strongly recommend that you **disable** [public pipelines](../../../ci/pipelines/settings.md#change-pipeline-visibility-for-non-project-members-in-public-projects) by setting the artifact's public flag to false (`public: false`). This setting ensures artifacts are accessible only to GitLab Administrators and project members with the Reporter role and above. @@ -119,7 +117,7 @@ inconsistent. Instead, use a remote storage resource. 1. Copy a pre-populated Terraform `init` command: 1. On the top bar, select **Main menu > Projects** and find your project. - 1. On the left sidebar, select **Infrastructure > Terraform**. + 1. On the left sidebar, select **Infrastructure > Terraform states**. 1. Next to the environment you want to use, select **Actions** (**{ellipsis_v}**) and select **Copy Terraform init command**. @@ -297,7 +295,7 @@ To read the Terraform state in the target project, you need at least the Develop To view Terraform state files: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Infrastructure > Terraform**. +1. On the left sidebar, select **Infrastructure > Terraform states**. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4563) to track improvements to this UI. @@ -323,7 +321,7 @@ curl --header "Private-Token: <your_access_token>" --request DELETE "https://git If you have at least the Maintainer role, you can remove a state file. -1. On the left sidebar, select **Infrastructure > Terraform**. +1. On the left sidebar, select **Infrastructure > Terraform states**. 1. In the **Actions** column, select **Actions** (**{ellipsis_v}**) and then **Remove state file and versions**. ### Remove a state file by using the API @@ -338,6 +336,5 @@ You can also use [the GraphQL API](../../../api/graphql/reference/index.md#mutat ## Related topics -- [Troubleshooting GitLab-managed Terraform state](troubleshooting.md). -- To use GitLab and Terraform to deploy an AWS EC2 instance in a custom VPC, - see [this sample project](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws). +- [Troubleshooting GitLab-managed Terraform state](troubleshooting.md) +- [Sample project: Terraform deployment of AWS EC2 instance in a custom VPC](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws) diff --git a/doc/user/infrastructure/iac/terraform_template_recipes.md b/doc/user/infrastructure/iac/terraform_template_recipes.md index 89a97f305e4..0011e7c9242 100644 --- a/doc/user/infrastructure/iac/terraform_template_recipes.md +++ b/doc/user/infrastructure/iac/terraform_template_recipes.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -36,7 +36,7 @@ include: - template: Terraform.latest.gitlab-ci.yml deploy: - envrionment: + environment: name: $TF_STATE_NAME action: start on_stop: destroy @@ -65,7 +65,7 @@ build: - when: on_success deploy: - envrionment: + environment: name: $TF_STATE_NAME action: start on_stop: destroy diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md index 2aa1e5d3284..d770c0111d0 100644 --- a/doc/user/infrastructure/iac/troubleshooting.md +++ b/doc/user/infrastructure/iac/troubleshooting.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -160,12 +160,3 @@ If your `TF_HTTP_ADDRESS`, `TF_HTTP_LOCK_ADDRESS` and `TF_HTTP_UNLOCK_ADDRESS` a to update the state names there. Alternatively, you can [migrate your terraform state](terraform_state.md#migrate-to-a-gitlab-managed-terraform-state). - -#### Self-managed GitLab instances - -By default, support for state names with periods is not enabled on self-managed GitLab. -You can enable it from the Rails console: - -```ruby -Feature.enable(:allow_dots_on_tf_state_names) -``` diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 87633932e0d..d60f20a3713 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md index 88430df0d67..dc5d402d04b 100644 --- a/doc/user/instance/clusters/index.md +++ b/doc/user/instance/clusters/index.md @@ -4,7 +4,7 @@ group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Instance-level Kubernetes clusters (certificate-based) (DEPRECATED) **(FREE SELF)** +# Instance-level Kubernetes clusters (certificate-based) (deprecated) **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in GitLab 11.11. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/markdown.md b/doc/user/markdown.md index eaceeccc148..b8ed1c06324 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -124,8 +124,13 @@ to see the color chips next to the color code: ### Diagrams and flowcharts -You can generate diagrams and flowcharts from text by using [Mermaid](https://mermaidjs.github.io/) or [PlantUML](https://plantuml.com). -You can also use [Kroki](https://kroki.io) to create a wide variety of diagrams. +You can generate diagrams from text by using: + +- [Mermaid](https://mermaidjs.github.io/) +- [PlantUML](https://plantuml.com) +- [Kroki](https://kroki.io) to create a wide variety of diagrams. + +In wikis, you can also add and edit diagrams created with the [diagrams.net editor](#diagramsnet-editor). #### Mermaid @@ -206,27 +211,44 @@ For more information, see the [Kroki integration](../administration/integration/ [View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#emojis). -```markdown -Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: +::Tabs -:zap: You can use emoji anywhere GitLab Flavored Markdown is supported. :v: +:::TabTitle Rendered Markdown -You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People :heart: you for that. +<!-- vale gitlab.Markdown_emoji = NO --> -If you're new to this, don't be :fearful:. You can join the emoji :family:. Just look up one of the supported codes. +Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/monkey.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":monkey:" alt=":monkey:"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/star2.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":star2:" alt=":star2:"> to your <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":speech_balloon:" alt=":speech_balloon:">. Well we have a gift for you: -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. :thumbsup: -``` +<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/zap.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":zap:" alt=":zap:">You can use emoji anywhere GitLab Flavored Markdown is supported. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/v.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":v:" alt=":v:"> + +You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/bug.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":bug:" alt=":bug:"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":speak_no_evil:" alt=":speak_no_evil:"> patches. If someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/snail.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":snail:" alt=":snail:"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/birthday.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":birthday:" alt=":birthday:">. People <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/heart.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":heart:" alt=":heart:"> you for that. -Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/monkey.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/star2.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. Well we have a gift for you: +If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/fearful.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":fearful:" alt=":fearful:">. You can join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/family.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":family:" alt=":family:">. Just look up one of the supported codes. -<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/zap.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">You can use emoji anywhere GitLab Flavored Markdown is supported. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/v.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0;border:0;padding:0;" title=":thumbsup:" alt=":thumbsup:"> -You can use it to point out a<img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/bug.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> patches. If someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/snail.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/birthday.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. People <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/heart.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> you for that. +<!-- vale gitlab.Markdown_emoji = YES --> -If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/fearful.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. You can join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/family.png" width="20px" height="20px" style="display:inline;margin:0;border: 0">. Just look up one of the supported codes. +:::TabTitle Code -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/public/-/emojis/2/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0;border: 0"> +```markdown +Sometimes you want to :monkey: around a bit and add some :star2: to your +:speech_balloon:. Well we have a gift for you: + +:zap: You can use emoji anywhere GitLab Flavored Markdown is supported. :v: + +You can use it to point out a :bug: or warn about :speak_no_evil: patches. +And if someone improves your really :snail: code, send them some :birthday:. +People :heart: you for that. + +If you're new to this, don't be :fearful:. You can join the emoji :family:. +Just look up one of the supported codes. + +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) +for a list of all supported emoji codes. :thumbsup: +``` + +::EndTabs #### Emojis and your operating system @@ -244,6 +266,8 @@ this font installed by default. <!-- vale gitlab.Spelling = YES --> +To learn more about adding custom emojis, see [Custom emojis](award_emojis.md#custom-emojis). + ### Front matter Front matter is metadata included at the beginning of a Markdown document, preceding @@ -357,6 +381,10 @@ _KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX. This syntax also works for the Asciidoctor `:stem: latexmath`. For details, see the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). +To prevent malicious activity, GitLab renders only the first 50 inline math instances. +The number of math blocks is also limited based on render time. If the limit is exceeded, +GitLab renders the excess math instances as text. + Math written between dollar signs with backticks (``$`...`$``) or single dollar signs (`$...$`) is rendered inline with the text. @@ -366,15 +394,15 @@ the language declared as `math` is rendered on a separate line: ````markdown This math is inline: $`a^2+b^2=c^2`$. -This math is on a separate line: +This math is on a separate line using a ```` ```math ```` block: ```math a^2+b^2=c^2 ``` -This math is on a separate line: $$a^2+b^2=c^2$$ +This math is on a separate line using inline `$$`: $$a^2+b^2=c^2$$ -This math is on a separate line: +This math is on a separate line using a `$$...$$` block: $$ a^2+b^2=c^2 @@ -383,23 +411,15 @@ $$ This math is inline: $`a^2+b^2=c^2`$. -This math is on a separate line: +This math is on a separate line using a ```` ```math ```` block: ```math a^2+b^2=c^2 ``` -This math is on a separate line: $$a^2+b^2=c^2$$ - -This math is on a separate line: - -$$ -a^2+b^2=c^2 -$$ - -This math is on a separate line: $$a^2+b^2=c^2$$ +This math is on a separate line using inline `$$`: $$a^2+b^2=c^2$$ -This math is on a separate line: +This math is on a separate line using a `$$...$$` block: $$ a^2+b^2=c^2 @@ -543,6 +563,58 @@ This example links to `<wiki_root>/miscellaneous.md`: [Link to Related Page](/miscellaneous.md) ``` +#### diagrams.net editor + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322174) in GitLab 15.10. + +NOTE: +Use of the diagrams.net editor is not available on offline environments. + +In wikis, you can use the [diagrams.net](https://www.diagrams.net/) editor to create diagrams. You +can also edit diagrams created with the diagrams.net editor. The diagram editor is available in both +the Markdown editor and the content editor. + +##### Markdown editor + +To create a diagram in the Markdown editor: + +1. In the editor's toolbar, select **Insert or edit diagram** (**{diagram}**). +1. Use the diagrams.net editor to create the diagram. +1. Select **Save & exit**. + +A Markdown image reference to the diagram is inserted in the wiki content. + +To edit a diagram in the Markdown editor: + +1. Place the Markdown editor's text field cursor in a Markdown image reference +that contains the diagram. +1. Select **Insert or edit diagram** (**{diagram}**) in the Markdown editor. +1. Use the diagrams.net editor to edit the diagram. +1. Select **Save & exit**. + +A Markdown image reference to the diagram is inserted in the wiki content, +replacing the previous diagram. + +##### Content editor + +To create a diagram in the content editor: + +1. In the editor's toolbar, select **More options** (**{plus}**). +1. In the dropdown list, select **Create or edit diagram**. +1. Use the diagrams.net editor to create the diagram. +1. Select **Save & exit**. + +The diagram as visualized in the diagrams.net editor is inserted in the wiki content. + +To edit a diagram in the content editor: + +1. Select the diagram that you want to edit. +1. In the floating toolbar, select **Edit diagram** (**{diagram}**). +1. Use the diagrams.net editor to edit the diagram. +1. Select **Save & exit**. + +The selected diagram is replaced with an updated version. + ## GitLab-specific references GitLab Flavored Markdown renders GitLab-specific references. For example, you can reference @@ -597,19 +669,44 @@ In addition to this, links to some objects are also recognized and formatted. So ### Show the issue, merge request, or epic title in the reference -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15694) in GitLab 14.6. +> - Support for issues, merge requests, and epics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15694) in GitLab 14.6. +> - Support for work items [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390854) in GitLab 16.0. -To include the title in the rendered link of an issue, merge request, or epic, add a plus (`+`) +To include the title in the rendered link of an issue, work item, merge request, or epic, add a plus (`+`) at the end of the reference. For example, a reference like `#123+` is rendered as `The issue title (#123)`. URL references like `https://gitlab.com/gitlab-org/gitlab/-/issues/1234+` are also expanded. -### Embedding metrics in GitLab Flavored Markdown +### Show the issue, work item or merge request summary in the reference + +> - Support for issues and merge requests [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386937) in GitLab 15.10. +> - Support for work items [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390854) in GitLab 16.0. + +To include an extended summary in the rendered link of an issue, work item, or merge request, add a `+s` +at the end of the reference. Summary includes information about **assignees**, **milestone** +and **health status** of referenced item. + +For example, a reference like `#123+s` is rendered as +`The issue title (#123) • First Assignee, Second Assignee+ • v15.10 • Needs attention`. + +URL references like `https://gitlab.com/gitlab-org/gitlab/-/issues/1234+s` are also expanded. + +### Embedding metrics Metric charts can be embedded in GitLab Flavored Markdown. Read [Embedding Metrics in GitLab flavored Markdown](../operations/metrics/embed.md) for more details. +### Embedding Observability dashboards + +You can embed GitLab Observability UI dashboards descriptions and comments, for example in epics, issues, and MRs. + +To embed an Observability dashboard URL: + +1. In GitLab Observability UI, copy the URL in the address bar. + +1. Paste your link wherever you want to embed your dashboard. GitLab Flavored Markdown recognizes the URL and displays the source. + ## Features extended from standard Markdown All standard Markdown formatting should work as expected in GitLab. Some standard @@ -656,13 +753,17 @@ you can quote that without having to manually prepend `>` to every line! ``` ->>> -If you paste a message from somewhere else - -that spans multiple lines, +<!-- +Use a standard blockquote here until https://gitlab.com/gitlab-org/gitlab/-/issues/390290 +gets properly fixed. The mixture of HTML comments and HTML tags +trigger this problem. +--> -you can quote that without having to manually prepend `>` to every line! ->>> +> If you paste a message from somewhere else +> +> that spans multiple lines, +> +> you can quote that without having to manually prepend `>` to every line! ### Code spans and blocks @@ -727,7 +828,7 @@ Tildes are OK too. If this section isn't rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). -GitLab uses the [Rouge Ruby library](http://rouge.jneen.net/) for more colorful syntax +GitLab uses the [Rouge Ruby library](https://github.com/rouge-ruby/rouge) for more colorful syntax highlighting in code blocks. For a list of supported languages visit the [Rouge project wiki](https://github.com/rouge-ruby/rouge/wiki/List-of-supported-languages-and-lexers). Syntax highlighting is supported only in code blocks, so you can't highlight inline code. @@ -1594,7 +1695,7 @@ use `<br>` tags to force a cell to have multiple lines: | Item2 | This item has:<br>- Multiple items<br>- That we want listed separately | You can use HTML formatting in GitLab itself to add [task lists](#task-lists) with checkboxes, -but they do not render properly on `docs.gitlab.com`. Note that these tasks will not save their +but they do not render properly on `docs.gitlab.com`. These tasks will not save their state when selected, like regular GitLab task lists. ```markdown diff --git a/doc/user/namespace/index.md b/doc/user/namespace/index.md index 7463b8f1099..00eb63da3ff 100644 --- a/doc/user/namespace/index.md +++ b/doc/user/namespace/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/okrs.md b/doc/user/okrs.md index 0d3be8474fe..030f6eb82ef 100644 --- a/doc/user/okrs.md +++ b/doc/user/okrs.md @@ -8,8 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103355) in GitLab 15.6 [with a flag](../administration/feature_flags.md) named `okrs_mvc`. Disabled by default. -WARNING: -OKRs are in [**Alpha**](../policy/alpha-beta-support.md#alpha-features). +OKRs are an [Experiment](../policy/alpha-beta-support.md#experiment). For the OKR feature roadmap, see [epic 7864](https://gitlab.com/groups/gitlab-org/-/epics/7864). FLAG: @@ -17,12 +16,11 @@ On self-managed GitLab, by default this feature is not available. To make it ava On GitLab.com, this feature is not available. The feature is not ready for production use. -Use objectives and key results to align your workforce towards common goals and track the progress. -Set a big goal with an objective and use [child objectives and key results](#child-objectives-and-key-results) -to measure the big goal's completion. +[Objectives and key results](https://en.wikipedia.org/wiki/OKR) (OKRs) are a framework for setting +and tracking goals that are aligned with your organization's overall strategy and vision. The objective and the key result in GitLab share many features. In the documentation, the term -**OKR** refers to either an objective or a key result. +**OKRs** refers to both objectives and key results. OKRs are a type of work item, a step towards [default issue types](https://gitlab.com/gitlab-org/gitlab/-/issues/323404) in GitLab. @@ -31,6 +29,29 @@ to work items and adding custom work item types, see [epic 6033](https://gitlab.com/groups/gitlab-org/-/epics/6033) or the [Plan direction page](https://about.gitlab.com/direction/plan/). +## Designing effective OKRs + +Use objectives and key results to align your workforce towards common goals and track the progress. +Set a big goal with an objective and use [child objectives and key results](#child-objectives-and-key-results) +to measure the big goal's completion. + +**Objectives** are aspirational goals to be achieved and define **what you're aiming to do**. +They show how an individual's, team's, or department's work impacts overall direction of the +organization by connecting their work to overall company strategy. + +**Key results** are measures of progress against aligned objectives. They express +**how you know if you have reached your goal** (objective). +By achieving a specific outcome (key result), you create progress for the linked objective. + +To know if your OKR makes sense, you can use this sentence: + +<!-- vale gitlab.FutureTense = NO --> +> I/we will accomplish (objective) by (date) through attaining and achieving the following metrics (key results). +<!-- vale gitlab.FutureTense = YES --> + +To learn how to create better OKRs and how we use them at GitLab, see the +[Objectives and Key Results handbook page](https://about.gitlab.com/company/okrs/). + ## Create an objective Prerequisites: @@ -93,12 +114,31 @@ To edit an OKR: 1. Optional. To edit the description, select the edit icon (**{pencil}**), make your changes, and select **Save**. +## View OKR system notes + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) to feature flag named `work_items_mvc` in GitLab 15.8. Disabled by default. +> - Changing activity sort order [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.8. +> - Filtering activity [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/389971) in GitLab 15.10. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.10. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +You can view all the system notes related to the task. By default they are sorted by **Oldest first**. +You can always change the sorting order to **Newest first**, which is remembered across sessions. + +## Comments and threads + +You can add [comments](discussions/index.md) and reply to threads in tasks. + ## Assign users To show who is responsible for an OKR, you can assign users to it. Users on GitLab Free can assign one user per OKR. -Users on GitLab Premium and higher can assign multiple users to a single OKR. +Users on GitLab Premium and Ultimate can assign multiple users to a single OKR. See also [multiple assignees for issues](project/issues/multiple_assignees_for_issues.md). Prerequisites: @@ -180,6 +220,20 @@ To set health status of an OKR: 1. [Open the key result](okrs.md#view-a-key-result) that you want to edit. 1. Next to **Health status**, select the dropdown list and select the desired health status. +## Promote a key result to an objective + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386877) in GitLab 16.0. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To promote a key result: + +1. [Open the key result](#view-a-key-result). +1. In the top right corner, select the vertical ellipsis (**{ellipsis_v}**).. +1. Select **Promote to objective**. + ## Close an OKR When an OKR is achieved, you can close it. @@ -252,3 +306,14 @@ To add an existing key result to an objective: To add multiple objectives, repeat this step. 1. Select **Add key result**. + +### Reorder objective and key result children + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385887) in GitLab 16.0. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +By default, child OKRs are ordered by creation date. +To reorder them, drag them around. diff --git a/doc/user/operations_dashboard/index.md b/doc/user/operations_dashboard/index.md index 2911a700e14..d013e2813f7 100644 --- a/doc/user/operations_dashboard/index.md +++ b/doc/user/operations_dashboard/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/organization/index.md b/doc/user/organization/index.md new file mode 100644 index 00000000000..f5af26250f6 --- /dev/null +++ b/doc/user/organization/index.md @@ -0,0 +1,42 @@ +--- +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Organization + +DISCLAIMER: +This page contains information related to upcoming products, features, and functionality. +It is important to note that the information presented is for informational purposes only. +Please do not rely on this information for purchasing or planning purposes. +As with all projects, the items mentioned on this page are subject to change or delay. +The development, release, and timing of any products, features, or functionality remain at the +sole discretion of GitLab Inc. + +NOTE: +Organization is in development. + +Organization will be above the [top-level namespaces](../namespace/index.md) for you to manage +everything you do as a GitLab administrator, including: + +- Defining and applying settings to all of your groups, subgroups, and projects. +- Aggregating data from all your groups, subgroups, and projects. + +Our goal is to reach feature parity between SaaS and self-managed installations, with all +[Admin Area settings](/ee/user/admin_area/settings/index.md) moving to either: + +- Groups. Available in the Organization, and subgroups. +- Hardware Controls. For functionality that does not apply to groups, Hardware Controls are only + applicable to self-managed installations. There is one Hardware Controls section per installation. + +For more information about the state of organization development, +see [epic 9265](https://gitlab.com/groups/gitlab-org/-/epics/9265). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video introduction to the new hierarchy concept for groups and projects for epics, see +[Consolidating groups and projects update (August 2021)](https://www.youtube.com/watch?v=fE74lsG_8yM). + +## Related topics + +- [Organization developer documentation](../../development/organization/index.md) diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 05daa525893..f06a7d83f92 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -315,8 +315,17 @@ To search by full or partial package name, or by exact recipe, run the conan search He* --remote=gitlab ``` -The scope of your search includes all projects you have permission to access. -This includes your private projects as well as all public projects. +The scope of your search depends on your Conan remote configuration: + +- If you have a remote configured for your [instance](#add-a-remote-for-your-instance), your search includes +all projects you have permission to access. This includes your private projects + as well as all public projects. + +- If you have a remote configured for a [project](#add-a-remote-for-your-project), your search includes all +packages in the target project, as long as you have permission to access it. + +NOTE: +The limit of the search results is 500 packages, and the results are sorted by the most recently published packages. ## Fetch Conan package information from the Package Registry diff --git a/doc/user/packages/container_registry/authenticate_with_container_registry.md b/doc/user/packages/container_registry/authenticate_with_container_registry.md index f48ba7bbf52..31337d19d59 100644 --- a/doc/user/packages/container_registry/authenticate_with_container_registry.md +++ b/doc/user/packages/container_registry/authenticate_with_container_registry.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w <!--- start_remove The following content will be removed on remove_date: '2023-11-22' --> WARNING: -[External authorization](../../admin_area/settings/external_authorization.md) will be enabled by default in GitLab 16.0. External authorization prevents personal access tokens and deploy tokens from accessing container and package registries and affects all users who use these tokens to access the registries. You can disable external authorization if you want to use personal access tokens and deploy tokens with the container or package registries. +In GitLab 16.0 and later, [external authorization](../../admin_area/settings/external_authorization.md) prevents personal access tokens and deploy tokens from accessing container and package registries and affects all users who use these tokens to access the registries. You can disable external authorization if you want to use personal access tokens and deploy tokens with the container or package registries. <!--- end_remove --> To authenticate with the Container Registry, you can use a: diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 4d7b25e2d77..c27265ccc3f 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -143,7 +143,7 @@ see [Container Registry visibility permissions](#container-registry-visibility-p is internal or private, the Container Registry is also internal or private. - **Only Project Members**: The Container Registry is visible only to project members with - Reporter role or higher. This visibility is similar to the behavior of a private project with Container + at least the Reporter role. This visibility is similar to the behavior of a private project with Container Registry visibility set to **Everyone With Access**. 1. Select **Save changes**. diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index 9229ea61821..f873453e049 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -21,7 +21,7 @@ The Usage Quotas page (**Settings > Usage Quotas > Storage**) displays storage u This page includes the Container Registry usage, which is only available on GitLab.com. Measuring usage is only possible on the new version of the GitLab Container Registry backed by a metadata database. Support for improvements is proposed in epic [5523](https://gitlab.com/groups/gitlab-org/-/epics/5523). -You cannot use the Container Registry in self-managed instances, but epic [5521](https://gitlab.com/groups/gitlab-org/-/epics/5521) proposes to change this behavior. +On self-managed instances, you cannot use the Container Registry with a metadata database. For the plan to change this behavior, see [epic 5521](https://gitlab.com/groups/gitlab-org/-/epics/5521). Image layers stored in the Container Registry are deduplicated at the root namespace level. @@ -112,6 +112,43 @@ to work with all third-party registries in the same predictable way. If you use Registry, this workaround is not required because we implemented a special tag delete operation. In this case, you can expect cleanup policies to be consistent and predictable. +#### Example cleanup policy workflow + +The interaction between the keep and remove rules for the cleanup policy can be complex. +For example, with a project with this cleanup policy configuration: + +- **Keep the most recent**: 1 tag per image name. +- **Keep tags matching**: `production-.*` +- **Remove tags older than**: 7 days. +- **Remove tags matching**: `.*`. + +And a container repository with these tags: + +- `latest`, published 2 hours ago. +- `production-v44`, published 3 days ago. +- `production-v43`, published 6 days ago. +- `production-v42`, published 11 days ago. +- `dev-v44`, published 2 days ago. +- `dev-v43`, published 5 day ago. +- `dev-v42`, published 10 days ago. +- `v44`, published yesterday. +- `v43`, published 12 days ago. +- `v42`, published 20 days ago. + +In this example, the tags that would be deleted in the next cleanup run are `dev-v42`, `v43`, and `v42`. +You can interpret the rules as applying with this precedence: + +1. The keep rules have highest precedence. Tags must be kept when they match **any** rule. + - The `latest` tag must be kept, because `latest` tags are always kept. + - The `production-v44`, `production-v43`, and `production-v42` tags must be kept, + because they match the **Keep tags matching** rule. + - The `v44` tag must be kept because it's the most recent, matching the **Keep the most recent** rule. +1. The remove rules have lower precedence, and tags are only deleted if **all** rules match. + For the tags not matching any keep rules (`dev-44`, `dev-v43`, `dev-v42`, `v43`, and `v42`): + - `dev-44` and `dev-43` do **not** match the **Remove tags older than**, and are kept. + - `dev-v42`, `v43`, and `v42` match both **Remove tags older than** and **Remove tags matching** + rules, so these three tags can be deleted. + ### Create a cleanup policy You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the UI. @@ -119,8 +156,8 @@ You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the To create a cleanup policy in the UI: 1. For your project, go to **Settings > Packages and registries**. -1. Expand the **Clean up image tags** section. -1. Complete the fields. +1. In the **Cleanup policies** section, select **Set cleanup rules**. +1. Complete the fields: | Field | Description | |----------------------------|-------------------------------------------------| @@ -281,7 +318,7 @@ Here are some other options you can use to reduce the Container Registry storage If you see this error message, check the regex patterns to ensure they are valid. -GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions in the cleanup policy. You can test them with the [regex101 regex tester](https://regex101.com/). +GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions in the cleanup policy. You can test them with the [regex101 regex tester](https://regex101.com/) using the `Golang` flavor. View some common [regex pattern examples](#regex-pattern-examples). ### The cleanup policy doesn't delete any tags diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md index 7ec20e3d036..220e2085637 100644 --- a/doc/user/packages/debian_repository/index.md +++ b/doc/user/packages/debian_repository/index.md @@ -11,8 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Deployed behind a feature flag](../../feature_flags.md), disabled by default. WARNING: -The Debian package registry for GitLab is under development and isn't ready for production use due to -limited functionality. This [epic](https://gitlab.com/groups/gitlab-org/-/epics/6057) details the remaining +The Debian package registry for GitLab is under development and isn't ready for production use. This [epic](https://gitlab.com/groups/gitlab-org/-/epics/6057) details the remaining work and timelines to make it production ready. Publish Debian packages in your project's Package Registry. Then install the @@ -23,6 +22,12 @@ Project and Group packages are supported. For documentation of the specific API endpoints that Debian package manager clients use, see the [Debian API documentation](../../../api/packages/debian.md). +Prerequisites: + +- The `dpkg-deb` binary must be installed on the GitLab instance. + This binary is usually provided by the [`dpkg` package](https://wiki.debian.org/Teams/Dpkg/Downstream), + installed by default on Debian and derivatives. + ## Enable the Debian API **(FREE SELF)** Debian repository support is still a work in progress. It's gated behind a feature flag that's @@ -30,6 +35,9 @@ Debian repository support is still a work in progress. It's gated behind a featu [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to enable it. +WARNING: +Understand the [stability and security risks of enabling features still in development](../../../administration/feature_flags.md#risks-when-enabling-features-still-in-development). + To enable it: ```ruby @@ -46,6 +54,9 @@ Feature.disable(:debian_packages) The Debian group repository is also behind a second feature flag that is disabled by default. +WARNING: +Understand the [stability and security risks of enabling features still in development](../../../administration/feature_flags.md#risks-when-enabling-features-still-in-development). + To enable it: ```ruby @@ -92,8 +103,11 @@ with one of the following: ## Create a Distribution -On the project-level, Debian packages are published using *Debian Distributions*. To publish -packages on the group level, create a distribution with the same `codename`. +At the project level, Debian packages are published with **Debian distributions**. At the +group level, Debian packages are aggregated from the projects in the group provided that: + +- The project visibility is set to `public`. +- The Debian `codename` for the group matches the Debian `codename` for the project. To create a project-level distribution using a personal access token: @@ -135,6 +149,7 @@ Once built, several files are created: - `.deb` files: the binary packages - `.udeb` files: lightened .deb files, used for Debian-Installer (if needed) +- `.ddeb` files: Ubuntu debug .deb files (if needed) - `.tar.{gz,bz2,xz,...}` files: Source files - `.dsc` file: Source metadata, and list of source files (with hashes) - `.buildinfo` file: Used for Reproducible builds (optional) @@ -155,9 +170,9 @@ EOF dput --config=dput.cf --unchecked --no-upload-log gitlab <your_package>.changes ``` -## Directly upload a package +## Upload a package with explicit distribution and component -> Direct upload [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101838) in GitLab 15.9. +> Upload with explicit distribution and component [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101838) in GitLab 15.9. When you don't have access to `.changes` file, you can directly upload a `.deb` by passing distribution `codename` and target `component` as parameters with @@ -166,7 +181,8 @@ For example, to upload to component `main` of distribution `sid` using a persona ```shell curl --request PUT --user "<username>:<personal_access_token>" \ - "https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian/?distribution=sid&component=main" \ + --get --data "distribution=sid" --data "component=main" \ + "https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian/" \ --upload-file /path/to/your.deb ``` @@ -176,39 +192,39 @@ To install a package: 1. Configure the repository: - If you are using a private project, add your [credentials](#authenticate-to-the-debian-package-repositories) to your apt configuration: + If you are using a private project, add your [credentials](#authenticate-to-the-debian-package-repositories) to your apt configuration: - ```shell - echo 'machine gitlab.example.com login <username> password <password>' \ - | sudo tee /etc/apt/auth.conf.d/gitlab_project.conf - ``` + ```shell + echo 'machine gitlab.example.com login <username> password <password>' \ + | sudo tee /etc/apt/auth.conf.d/gitlab_project.conf + ``` - Download your distribution key using your [credentials](#authenticate-to-the-debian-distributions-apis): + Download your distribution key using your [credentials](#authenticate-to-the-debian-distributions-apis): - ```shell - sudo mkdir -p /usr/local/share/keyrings - curl --header "PRIVATE-TOKEN: <your_access_token>" \ - "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions/<codename>/key.asc" \ - | \ - gpg --dearmor \ - | \ - sudo tee /usr/local/share/keyrings/<codename>-archive-keyring.gpg \ - > /dev/null - ``` + ```shell + sudo mkdir -p /usr/local/share/keyrings + curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions/<codename>/key.asc" \ + | \ + gpg --dearmor \ + | \ + sudo tee /usr/local/share/keyrings/<codename>-archive-keyring.gpg \ + > /dev/null + ``` - Add your project as a source: + Add your project as a source: - ```shell - echo 'deb [ signed-by=/usr/local/share/keyrings/<codename>-archive-keyring.gpg ] https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian <codename> <component1> <component2>' \ - | sudo tee /etc/apt/sources.list.d/gitlab_project.list - sudo apt-get update - ``` + ```shell + echo 'deb [ signed-by=/usr/local/share/keyrings/<codename>-archive-keyring.gpg ] https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian <codename> <component1> <component2>' \ + | sudo tee /etc/apt/sources.list.d/gitlab_project.list + sudo apt-get update + ``` 1. Install the package: - ```shell - sudo apt-get -y install -t <codename> <package-name> - ``` + ```shell + sudo apt-get -y install -t <codename> <package-name> + ``` ## Download a source package @@ -216,36 +232,36 @@ To download a source package: 1. Configure the repository: - If you are using a private project, add your [credentials](#authenticate-to-the-debian-package-repositories) to your apt configuration: + If you are using a private project, add your [credentials](#authenticate-to-the-debian-package-repositories) to your apt configuration: - ```shell - echo 'machine gitlab.example.com login <username> password <password>' \ - | sudo tee /etc/apt/auth.conf.d/gitlab_project.conf - ``` + ```shell + echo 'machine gitlab.example.com login <username> password <password>' \ + | sudo tee /etc/apt/auth.conf.d/gitlab_project.conf + ``` - Download your distribution key using your [credentials](#authenticate-to-the-debian-distributions-apis): + Download your distribution key using your [credentials](#authenticate-to-the-debian-distributions-apis): - ```shell - sudo mkdir -p /usr/local/share/keyrings - curl --header "PRIVATE-TOKEN: <your_access_token>" \ - "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions/<codename>/key.asc" \ - | \ - gpg --dearmor \ - | \ - sudo tee /usr/local/share/keyrings/<codename>-archive-keyring.gpg \ - > /dev/null - ``` + ```shell + sudo mkdir -p /usr/local/share/keyrings + curl --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions/<codename>/key.asc" \ + | \ + gpg --dearmor \ + | \ + sudo tee /usr/local/share/keyrings/<codename>-archive-keyring.gpg \ + > /dev/null + ``` - Add your project as a source: + Add your project as a source: - ```shell - echo 'deb-src [ signed-by=/usr/local/share/keyrings/<codename>-archive-keyring.gpg ] https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian <codename> <component1> <component2>' \ - | sudo tee /etc/apt/sources.list.d/gitlab_project-sources.list - sudo apt-get update - ``` + ```shell + echo 'deb-src [ signed-by=/usr/local/share/keyrings/<codename>-archive-keyring.gpg ] https://gitlab.example.com/api/v4/projects/<project_id>/packages/debian <codename> <component1> <component2>' \ + | sudo tee /etc/apt/sources.list.d/gitlab_project-sources.list + sudo apt-get update + ``` 1. Download the source package: - ```shell - sudo apt-get source -t <codename> <package-name> - ``` + ```shell + sudo apt-get source -t <codename> <package-name> + ``` diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 8dc3c98795b..d7cf33cc2a4 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -342,13 +342,21 @@ Authenticating with credentials from job payload (GitLab Registry) Make sure you are using the expected authentication mechanism. -### "Not Found" error when pulling image +### `Not Found` or `404` error when pulling image -Docker errors similar to the following may indicate that the user running the build job doesn't have -a minimum of the Guest role in the specified Dependency Proxy group: +Errors like these might indicate that the user running the job doesn't have +a minimum of the Guest role in the Dependency Proxy group: -```plaintext -ERROR: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found +- ```plaintext + ERROR: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found -failed to solve with frontend dockerfile.v0: failed to create LLB definition: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found -``` + failed to solve with frontend dockerfile.v0: failed to create LLB definition: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found + ``` + +- ```plaintext + ERROR: Job failed: failed to pull image "gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest" with specified policies [always]: + Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:237:1s) + ``` + +For more information about the work to improve the error messages in similar cases to `Access denied`, +see [issue 354826](https://gitlab.com/gitlab-org/gitlab/-/issues/354826). diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 932de0bcde6..bad099e6733 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -22,6 +22,8 @@ do not support the other available mechanisms. The `user-id` is not checked and may be any value, and the `password` must be either a [personal access token](../../../api/rest/index.md#personalprojectgroup-access-tokens), a [CI/CD job token](../../../ci/jobs/ci_job_token.md), or a [deploy token](../../project/deploy_tokens/index.md). +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ## Publish a package file When you publish a package file, if the package does not exist, it is created. diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index 1a089cd82be..f0e10d73d08 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -74,7 +74,7 @@ go env -w GOPROXY='https://gitlab.example.com/api/v4/projects/1234/packages/go,h With this configuration, Go fetches dependencies in this order: 1. Go attempts to fetch from the project-specific Go proxy. -1. Go attempts to fetch from [proxy.golang.org](https://proxy.golang.org). +1. Go attempts to fetch from [`proxy.golang.org`](https://proxy.golang.org). 1. Go fetches directly with version control system operations (like `git clone`, `svn checkout`, and so on). diff --git a/doc/user/packages/gradle_repository/index.md b/doc/user/packages/gradle_repository/index.md index 4247c13297d..456acc0da59 100644 --- a/doc/user/packages/gradle_repository/index.md +++ b/doc/user/packages/gradle_repository/index.md @@ -1,372 +1,11 @@ --- -stage: Package -group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../maven_repository/index.md' +remove_date: '2023-08-09' --- -# Maven packages in the Package Registry **(FREE)** +This document was moved to [another location](../maven_repository/index.md). -Publish [Maven](https://maven.apache.org) artifacts in your project's Package Registry using Gradle. -Then, install the packages whenever you need to use them as a dependency. - -For documentation of the specific API endpoints that the Maven package manager -client uses, see the [Maven API documentation](../../../api/packages/maven.md). - -Learn how to build a [Gradle](../workflows/build_packages.md#gradle) package. - -## Publish to the GitLab Package Registry - -### Tokens - -You need a token to publish a package. Different tokens are available depending on what you're trying to -achieve. For more information, review the [guidance on tokens](../package_registry/index.md#authenticate-with-the-registry). - -- If your organization uses two-factor authentication (2FA), you must use a personal access token with the scope set to `api`. -- If you publish a package via CI/CD pipelines, you must use a CI job token. - -Create a token and save it to use later in the process. - -## Authenticate to the Package Registry with Gradle - -### Authenticate with a personal access token or deploy token in Gradle - -In [your `GRADLE_USER_HOME` directory](https://docs.gradle.org/current/userguide/directory_layout.html#dir:gradle_user_home), -create a file `gradle.properties` with the following content: - -```properties -gitLabPrivateToken=REPLACE_WITH_YOUR_TOKEN -``` - -Your token name depends on which token you use. - -| Token type | Token name | -| --------------------- | --------------- | -| Personal access token | `Private-Token` | -| Deploy token | `Deploy-Token` | - -Add a `repositories` section to your -[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) -file: - -```groovy -repositories { - maven { - url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" - name "GitLab" - credentials(HttpHeaderCredentials) { - name = 'REPLACE_WITH_TOKEN_NAME' - value = gitLabPrivateToken - } - authentication { - header(HttpHeaderAuthentication) - } - } -} -``` - -Or add it to your `build.gradle.kts` file if you are using Kotlin DSL: - -```kotlin -repositories { - maven { - url = uri("https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven") - name = "GitLab" - credentials(HttpHeaderCredentials::class) { - name = "REPLACE_WITH_TOKEN_NAME" - value = findProperty("gitLabPrivateToken") as String? - } - authentication { - create("header", HttpHeaderAuthentication::class) - } - } -} -``` - -### Authenticate with a CI job token in Gradle - -To authenticate with a CI job token, add a `repositories` section to your -[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) -file: - -```groovy -repositories { - maven { - url "${CI_API_V4_URL}/groups/<group>/-/packages/maven" - name "GitLab" - credentials(HttpHeaderCredentials) { - name = 'Job-Token' - value = System.getenv("CI_JOB_TOKEN") - } - authentication { - header(HttpHeaderAuthentication) - } - } -} -``` - -Or add it to your `build.gradle.kts` file if you are using Kotlin DSL: - -```kotlin -repositories { - maven { - url = uri("$CI_API_V4_URL/groups/<group>/-/packages/maven") - name = "GitLab" - credentials(HttpHeaderCredentials::class) { - name = "Job-Token" - value = System.getenv("CI_JOB_TOKEN") - } - authentication { - create("header", HttpHeaderAuthentication::class) - } - } -} -``` - -### Naming convention - -You can use one of three API endpoints to install a Maven package. You must publish a package to a project, but note which endpoint -you use to install the package. The option you choose determines the settings you add to your `pom.xml` file for publishing. - -The three endpoints are: - -- **Project-level**: Use when you have a few Maven packages that are not in the same GitLab group. -- **Group-level**: Use when installing packages from many different projects in the same GitLab group. GitLab does not guarantee the uniqueness of package names in the group. You can have two projects with the same package name and package version. As a result, GitLab serves whichever one is more recent. -- **Instance-level**: Use when installing many packages from different GitLab groups or in their own namespace. - -**Only packages with the same path as the project** are exposed by the instance-level endpoint. - -| Project | Package | Instance-level endpoint available | -| ------------------- | -------------------------------- | --------------------------------- | -| `foo/bar` | `foo/bar/1.0-SNAPSHOT` | Yes | -| `gitlab-org/gitlab` | `foo/bar/1.0-SNAPSHOT` | No | -| `gitlab-org/gitlab` | `gitlab-org/gitlab/1.0-SNAPSHOT` | Yes | - -#### Endpoint URLs - -| Endpoint | Endpoint URL | Additional information | -| -------- | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | -| Project | `https://gitlab.example.com/api/v4/projects/<project_id>/packages/maven` | Replace `gitlab.example.com` with your domain name. Replace `<project_id>` with your project ID found on your project's homepage. | -| Group | `https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/maven` | Replace `gitlab.example.com` with your domain name. Replace `<group_id>` with your group ID found on your group's homepage. | -| Instance | `https:///gitlab.example.com/api/v4/packages/maven` | Replace `gitlab.example.com` with your domain name. | - -In all cases, to publish a package, you need: - -- A project-specific URL in the `distributionManagement` section. -- A `repository` and `distributionManagement` section. - -### Edit the Groovy DSL or Kotlin DSL - -The Gradle Groovy DSL `repositories` section should look like this: - -```groovy -repositories { - maven { - url "<your_endpoint_url>" - name "GitLab" - } -} -``` - -In Kotlin DSL: - -```kotlin -repositories { - maven { - url = uri("<your_endpoint_url>") - name = "GitLab" - } -} -``` - -- Replace `<your_endpoint_url>` with the [endpoint](#endpoint-urls) you chose. - -## Publish using Gradle - -Your token name depends on which token you use. - -| Token type | Token name | -| --------------------- | --------------- | -| Personal access token | `Private-Token` | -| Deploy token | `Deploy-Token` | - -To publish a package by using Gradle: - -1. Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: - - In Groovy DSL: - - ```groovy - plugins { - id 'java' - id 'maven-publish' - } - ``` - - In Kotlin DSL: - - ```kotlin - plugins { - java - `maven-publish` - } - ``` - -1. Add a `publishing` section: - - In Groovy DSL: - - ```groovy - publishing { - publications { - library(MavenPublication) { - from components.java - } - } - repositories { - maven { - url "https://gitlab.example.com/api/v4/projects/<PROJECT_ID>/packages/maven" - credentials(HttpHeaderCredentials) { - name = "REPLACE_WITH_TOKEN_NAME" - value = gitLabPrivateToken // the variable resides in $GRADLE_USER_HOME/gradle.properties - } - authentication { - header(HttpHeaderAuthentication) - } - } - } - } - ``` - - In Kotlin DSL: - - ```kotlin - publishing { - publications { - create<MavenPublication>("library") { - from(components["java"]) - } - } - repositories { - maven { - url = uri("https://gitlab.example.com/api/v4/projects/<PROJECT_ID>/packages/maven") - credentials(HttpHeaderCredentials::class) { - name = "REPLACE_WITH_TOKEN_NAME" - value = - findProperty("gitLabPrivateToken") as String? // the variable resides in $GRADLE_USER_HOME/gradle.properties - } - authentication { - create("header", HttpHeaderAuthentication::class) - } - } - } - } - ``` - -1. Replace `PROJECT_ID` with your project ID, which you can find on your project's home page. - -1. Run the publish task: - - ```shell - gradle publish - ``` - -Go to your project's **Packages and registries** page and view the published packages. - -## Install a package - -To install a package from the GitLab Package Registry, you must configure -the [remote and authenticate](#authenticate-to-the-package-registry-with-gradle). -After configuring the remote and authenticate, you can install a package from a project, group, or namespace. - -If multiple packages have the same name and version, when you install -a package, the most recently-published package is retrieved. - -Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to `build.gradle` in the dependencies section: - -```groovy -dependencies { - implementation 'com.mycompany.mydepartment:my-project:1.0-SNAPSHOT' -} -``` - -Or to `build.gradle.kts` if you are using Kotlin DSL: - -```kotlin -dependencies { - implementation("com.mycompany.mydepartment:my-project:1.0-SNAPSHOT") -} -``` - -## Helpful hints - -For the complete list of helpful hints, see the [Maven documentation](../maven_repository/index.md#helpful-hints). - -### Create Maven packages with GitLab CI/CD by using Gradle - -You can create a package each time the `main` branch -is updated. - -1. Authenticate with [a CI job token in Gradle](#authenticate-with-a-ci-job-token-in-gradle). - -1. Add a `deploy` job to your `.gitlab-ci.yml` file: - - ```yaml - deploy: - image: gradle:6.5-jdk11 - script: - - 'gradle publish' - only: - - main - ``` - -1. Commit files to your repository. - -When the pipeline is successful, the Maven package is created. - -### Publishing a package with the same name or version - -When you publish a package with the same name and version as an existing package, the new package -files are added to the existing package. You can still use the UI or API to access and view the -existing package's older assets. - -Consider using the Packages API or the UI to delete older package versions. - -### Do not allow duplicate Maven packages - -To prevent users from publishing duplicate Maven packages, you can use the [GraphQl API](../../../api/graphql/reference/index.md#packagesettings) or the UI. - -In the UI: - -1. For your group, go to **Settings > Packages and registries**. -1. Expand the **Package Registry** section. -1. Turn on the **Do not allow duplicates** toggle. -1. Optional. To allow some duplicate packages, in the **Exceptions** box, enter a regex pattern that matches the names and/or versions of packages you want to allow. - -Your changes are automatically saved. - -### Request forwarding to Maven Central - -FLAG: -By default, this feature is not available for self-managed. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `maven_central_request_forwarding`. -This feature is not available for SaaS users. - -When a Maven package is not found in the Package Registry, the request is forwarded -to [Maven Central](https://search.maven.org/). - -When the feature flag is enabled, administrators can disable this behavior in the -[Continuous Integration settings](../../admin_area/settings/continuous_integration.md). - -There are many ways to configure your Maven project to request packages -in Maven Central from GitLab. Maven repositories are queried in a -[specific order](https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order). -By default, maven-central is usually checked first through the -[Super POM](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM), so -GitLab needs to be configured to be queried before maven-central. - -[Using GitLab as a mirror of the central proxy](../maven_repository/index.md#setting-gitlab-as-a-mirror-for-the-central-proxy) is one -way to force GitLab to be queried in place of maven-central. - -Maven forwarding is restricted to only the project level and -group level [endpoints](#naming-convention). The instance-level endpoint -has naming restrictions that prevent it from being used for packages that don't follow that convention and also -introduces too much security risk for supply-chain style attacks. +<!-- This redirect file can be deleted after <2023-08-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/packages/harbor_container_registry/index.md b/doc/user/packages/harbor_container_registry/index.md index 1ad5e2c2f05..6cea541a55d 100644 --- a/doc/user/packages/harbor_container_registry/index.md +++ b/doc/user/packages/harbor_container_registry/index.md @@ -18,7 +18,7 @@ You can view the Harbor Registry for a project or group. You can search, sort, and filter images on this page. You can share a filtered view by copying the URL from your browser. -At the project level, you can see **CLI Commands** in the upper right corner, where you can copy +At the project level, in the upper-right corner, you can see **CLI Commands** where you can copy corresponding commands to sign in, build images, and push images. **CLI Commands** is not shown at the group level. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 7418977e0d1..2084de58bdb 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -14,15 +14,9 @@ packages, which can be easily consumed as a dependency in downstream projects. The GitLab [Container Registry](container_registry/index.md) is a secure and private registry for container images. It's built on open source software and completely integrated within GitLab. Use GitLab CI/CD to create and publish images. Use the GitLab [API](../../api/container_registry.md) to manage the registry across groups and projects. -## Infrastructure Registry +## Terraform Module Registry -The GitLab [Infrastructure Registry](infrastructure_registry/index.md) is a secure and private registry for infrastructure packages. You can use GitLab CI/CD to create and publish infrastructure packages. - -The Infrastructure Registry supports the following formats: - -| Package type | GitLab version | -| ------------ | -------------- | -| [Terraform Module](terraform_module_registry/index.md) | 14.0+ | +The GitLab [Terraform Module Registry](terraform_module_registry/index.md) is a secure and private registry for Terraform modules. You can use GitLab CI/CD to create and publish modules. ## Dependency Proxy diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md index d06c5e7fb1e..8c1e8a2ad8a 100644 --- a/doc/user/packages/infrastructure_registry/index.md +++ b/doc/user/packages/infrastructure_registry/index.md @@ -1,102 +1,11 @@ --- -stage: Package -group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../terraform_module_registry/index.md' +remove_date: '2023-06-13' --- -# Infrastructure Registry **(FREE)** +This document was moved to [another location](../terraform_module_registry/index.md). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. - -With the GitLab Infrastructure Registry, you can use GitLab projects as a -private registry for infrastructure packages. You can create and publish -packages with GitLab CI/CD, which can then be consumed from other private -projects. - -## View packages - -To view packages within your project: - -1. Go to the project. -1. Go to **Packages and registries > Infrastructure Registry**. - -You can search, sort, and filter packages on this page. - -For information on how to create and upload a package, view the GitLab -documentation for your package type: - -- [Terraform modules](../terraform_module_registry/index.md) - -## Use GitLab CI/CD to build packages - -To use [GitLab CI/CD](../../../ci/index.md) to build packages, you can -authenticate with the [`CI_JOB_TOKEN` predefined variable](../../../ci/variables/predefined_variables.md). - -CI/CD templates, which you can use to get started, are in [this repository](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). - -For more information about using CI/CD to build Terraform modules, -see [Publish a Terraform module by using CI/CD](../terraform_module_registry/index.md#publish-a-terraform-module-by-using-cicd). - -If you use CI/CD to build a package, you can find extended activity information -when you view the package details: - -![Package CI/CD activity](../package_registry/img/package_activity_v12_10.png) - -You can see the pipeline that published the package as well as the commit and the user who triggered it. However, the history is limited to five updates per package. - -## Download a package - -To download a package: - -1. Go to **Packages and registries > Infrastructure Registry**. -1. Select the name of the package you want to download. -1. In the **Activity** section, select the name of the package you want to download. - -## Delete a package - -You cannot edit a package after you publish it in the Infrastructure Registry. Instead, you -must delete and recreate it. - -To delete a package, you must have suitable [permissions](../../permissions.md). - -You can delete packages by using [the API](../../../api/packages.md#delete-a-project-package) or the UI. - -To delete a package in the UI, from your project: - -1. Go to **Packages and registries > Infrastructure Registry**. -1. Find the name of the package you want to delete. -1. Select **Delete**. - -The package is permanently deleted. - -## Disable the Infrastructure Registry - -The Infrastructure Registry is automatically enabled. - -For self-managed instances, a GitLab administrator can -[disable](../../../administration/packages/index.md) **Packages and registries**, -which removes this menu item from the sidebar. - -You can also remove the Infrastructure Registry for a specific project: - -1. In your project, go to **Settings > General**. -1. Expand the **Visibility, project features, permissions** section and toggle **Packages** off (in gray). -1. Select **Save changes**. - -To enable it back, follow the same steps above and toggle it on (in blue). - -## How module resolution works - -When you upload a new module, GitLab generates a path for the module, for example, `https://gitlab.example.com/parent-group/my-infra-package`. - -- This path conforms with [the Terraform spec](https://www.terraform.io/internals/module-registry-protocol). -- The name of the path must be unique within the namespace. - -For projects in subgroups, GitLab checks that the module name does not already exist anywhere in the namespace, including all subgroups and the parent group. - -For example, if: - -- The project is `gitlab.example.com/parent-group/sub-group/my-project`. -- The infrastructure package is `my-infra-package`. - -The project name must be unique in all projects in all groups under `parent-group`. +<!-- This redirect file can be deleted after <2023-06-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 1899cdc213f..fd8049d9b75 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -12,7 +12,13 @@ Then, install the packages whenever you need to use them as a dependency. For documentation of the specific API endpoints that the Maven package manager client uses, see the [Maven API documentation](../../../api/packages/maven.md). -Learn how to build a [Maven](../workflows/build_packages.md#maven) package. +Supported clients: + +- `mvn`. Learn how to build a [Maven](../workflows/build_packages.md#maven) package. +- `gradle`. Learn how to build a [Gradle](../workflows/build_packages.md#gradle) package. +- `sbt`. + - `sbt` can only be used to [pull dependencies](#install-a-package). + See this [issue 408479](https://gitlab.com/gitlab-org/gitlab/-/issues/408479) for more details. ## Publish to the GitLab Package Registry @@ -22,13 +28,16 @@ You need an token to publish a package. There are different tokens available dep Create a token and save it to use later in the process. -### Edit the `settings.xml` +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. -Add the following section to your -[`settings.xml`](https://maven.apache.org/settings.html) file. +#### Edit the client configuration -NOTE: -The `<name>` field must be named to match the token you chose. +You must add the authentication details to the configuration file +for your client. + +::Tabs + +:::TabTitle `mvn` | Token type | Name must be | Token | | --------------------- | --------------- | ---------------------------------------------------------------------- | @@ -36,6 +45,12 @@ The `<name>` field must be named to match the token you chose. | Deploy token | `Deploy-Token` | Paste token as-is, or define an environment variable to hold the token | | CI Job token | `Job-Token` | `${CI_JOB_TOKEN}` | +NOTE: +The `<name>` field must be named to match the token you chose. + +Add the following section to your +[`settings.xml`](https://maven.apache.org/settings.html) file. + ```xml <settings> <servers> @@ -54,6 +69,104 @@ The `<name>` field must be named to match the token you chose. </settings> ``` +:::TabTitle `gradle` + +| Token type | Name must be | Token | +| --------------------- | --------------- | ---------------------------------------------------------------------- | +| Personal access token | `Private-Token` | Paste token as-is, or define an environment variable to hold the token | +| Deploy token | `Deploy-Token` | Paste token as-is, or define an environment variable to hold the token | +| CI Job token | `Job-Token` | `System.getenv("CI_JOB_TOKEN")` | + +NOTE: +The `<name>` field must be named to match the token you chose. + +In [your `GRADLE_USER_HOME` directory](https://docs.gradle.org/current/userguide/directory_layout.html#dir:gradle_user_home), +create a file `gradle.properties` with the following content: + +```properties +gitLabPrivateToken=REPLACE_WITH_YOUR_TOKEN +``` + +Add a `repositories` section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +- In Groovy DSL: + + ```groovy + repositories { + maven { + url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'REPLACE_WITH_NAME' + value = gitLabPrivateToken + } + authentication { + header(HttpHeaderAuthentication) + } + } + } + ``` + +- In Kotlin DSL: + + ```kotlin + repositories { + maven { + url = uri("https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven") + name = "GitLab" + credentials(HttpHeaderCredentials::class) { + name = "REPLACE_WITH_NAME" + value = findProperty("gitLabPrivateToken") as String? + } + authentication { + create("header", HttpHeaderAuthentication::class) + } + } + } + ``` + +:::TabTitle `sbt` + +| Token type | Name must be | Token | +|-----------------------|------------------------------|------------------------------------------------------------------------| +| Personal access token | The username of the user | Paste token as-is, or define an environment variable to hold the token | +| Deploy token | The username of deploy token | Paste token as-is, or define an environment variable to hold the token | +| CI Job token | `gitlab-ci-token` | `sys.env.get("CI_JOB_TOKEN").get` | + +Authentication for [SBT](https://www.scala-sbt.org/index.html) is based on +[basic HTTP Authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication). +You must to provide a name and a password. + +NOTE: +The name field must be named to match the token you chose. + +To install a package from the Maven GitLab Package Registry by using `sbt`, you must configure +a [Maven resolver](https://www.scala-sbt.org/1.x/docs/Resolvers.html#Maven+resolvers). +If you're accessing a private or an internal project or group, you need to set up +[credentials](https://www.scala-sbt.org/1.x/docs/Publishing.html#Credentials). +After configuring the resolver and authentication, you can install a package +from a project, group, or namespace. + +In your [`build.sbt`](https://www.scala-sbt.org/1.x/docs/Directories.html#sbt+build+definition+files), add the following lines: + +```scala +resolvers += ("gitlab" at "<endpoint url>") + +credentials += Credentials("GitLab Packages Registry", "<host>", "<name>", "<token>") +``` + +In this example: + +- `<endpoint url>` is the [endpoint URL](#endpoint-urls). +Example: `https://gitlab.example.com/api/v4/projects/<project_id>/packages/maven`. +- `<host>` is the host present in the `<endpoint url>` without the protocol +scheme or the port. Example: `gitlab.example.com`. +- `<name>` and `<token>` are explained in the table above. + +::EndTabs + ### Naming convention You can use one of three endpoints to install a Maven package. You must publish a package to a project, but the endpoint you choose determines the settings you add to your `pom.xml` file for publishing. @@ -64,6 +177,13 @@ The three endpoints are: - **Group-level**: Use when you want to install packages from many different projects in the same GitLab group. GitLab does not guarantee the uniqueness of package names within the group. You can have two projects with the same package name and package version. As a result, GitLab serves whichever one is more recent. - **Instance-level**: Use when you have many packages in different GitLab groups or in their own namespace. +For the instance-level endpoint, ensure the relevant section of your `pom.xml` in Maven looks like this: + +```xml + <groupId>group-slug.subgroup-slug</groupId> + <artifactId>project-slug</artifactId> +``` + **Only packages that have the same path as the project** are exposed by the instance-level endpoint. | Project | Package | Instance-level endpoint available | @@ -80,7 +200,13 @@ The three endpoints are: | Group | `https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/maven` | Replace `gitlab.example.com` with your domain name. Replace `<group_id>` with your group ID, found on your group's homepage. | | Instance | `https://gitlab.example.com/api/v4/packages/maven` | Replace `gitlab.example.com` with your domain name. | -### Edit the `pom.xml` for publishing +### Edit the configuration file for publishing + +You must add publishing details to the configuration file for your client. + +::Tabs + +:::TabTitle `mvn` No matter which endpoint you choose, you must have: @@ -108,16 +234,97 @@ The relevant `repository` section of your `pom.xml` in Maven should look like th </distributionManagement> ``` -- The `id` is what you [defined in `settings.xml`](#edit-the-settingsxml). +- The `id` is what you [defined in `settings.xml`](#edit-the-client-configuration). - The `<your_endpoint_url>` depends on which [endpoint](#endpoint-urls) you choose. - Replace `gitlab.example.com` with your domain name. +:::TabTitle `gradle` + +To publish a package by using Gradle: + +1. Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: + + - In Groovy DSL: + + ```groovy + plugins { + id 'java' + id 'maven-publish' + } + ``` + + - In Kotlin DSL: + + ```kotlin + plugins { + java + `maven-publish` + } + ``` + +1. Add a `publishing` section: + + - In Groovy DSL: + + ```groovy + publishing { + publications { + library(MavenPublication) { + from components.java + } + } + repositories { + maven { + url "https://gitlab.example.com/api/v4/projects/<PROJECT_ID>/packages/maven" + credentials(HttpHeaderCredentials) { + name = "REPLACE_WITH_TOKEN_NAME" + value = gitLabPrivateToken // the variable resides in $GRADLE_USER_HOME/gradle.properties + } + authentication { + header(HttpHeaderAuthentication) + } + } + } + } + ``` + + - In Kotlin DSL: + + ```kotlin + publishing { + publications { + create<MavenPublication>("library") { + from(components["java"]) + } + } + repositories { + maven { + url = uri("https://gitlab.example.com/api/v4/projects/<PROJECT_ID>/packages/maven") + credentials(HttpHeaderCredentials::class) { + name = "REPLACE_WITH_TOKEN_NAME" + value = + findProperty("gitLabPrivateToken") as String? // the variable resides in $GRADLE_USER_HOME/gradle.properties + } + authentication { + create("header", HttpHeaderAuthentication::class) + } + } + } + } + ``` + +::EndTabs + ## Publish a package After you have set up the [authentication](#authenticate-to-the-package-registry) and [chosen an endpoint for publishing](#naming-convention), publish a Maven package to your project. +::Tabs + +:::TabTitle `mvn` + To publish a package by using Maven: ```shell @@ -138,6 +345,18 @@ The message should also show that the package was published to the correct locat Uploading to gitlab-maven: https://example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.jar ``` +:::TabTitle `gradle` + +Run the publish task: + +```shell +gradle publish +``` + +Go to your project's **Packages and registries** page and view the published packages. + +::EndTabs + ## Install a package To install a package from the GitLab Package Registry, you must configure @@ -148,7 +367,9 @@ group, or namespace. If multiple packages have the same name and version, when you install a package, the most recently-published package is retrieved. -### Use Maven with `mvn install` +::Tabs + +:::TabTitle `mvn` To install a package by using `mvn install`: @@ -175,9 +396,7 @@ The message should show that the package is downloading from the Package Registr Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -### Use Maven with `mvn dependency:get` - -You can install packages by using the Maven `dependency:get` [command](https://maven.apache.org/plugins/maven-dependency-plugin/get-mojo.html) directly. +You can also install packages by using the Maven [`dependency:get` command](https://maven.apache.org/plugins/maven-dependency-plugin/get-mojo.html) directly. 1. In your project directory, run: @@ -186,7 +405,7 @@ You can install packages by using the Maven `dependency:get` [command](https://m ``` - `<gitlab endpoint url>` is the URL of the GitLab [endpoint](#endpoint-urls). - - `<path to settings.xml>` is the path to the `settings.xml` file that contains the [authentication details](#edit-the-settingsxml). + - `<path to settings.xml>` is the path to the `settings.xml` file that contains the [authentication details](#edit-the-client-configuration). NOTE: The repository IDs in the command(`gitlab-maven`) and the `settings.xml` file must match. @@ -197,6 +416,52 @@ The message should show that the package is downloading from the Package Registr Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` +:::TabTitle `gradle` + +To install a package by using `gradle`: + +1. Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to `build.gradle` in the dependencies section: + + - In Groovy DSL: + + ```groovy + dependencies { + implementation 'com.mycompany.mydepartment:my-project:1.0-SNAPSHOT' + } + ``` + + - In Kotlin DSL: + + ```kotlin + dependencies { + implementation("com.mycompany.mydepartment:my-project:1.0-SNAPSHOT") + } + ``` + +1. In your project, run the following: + + ```shell + gradle install + ``` + +:::TabTitle `sbt` + +To install a package by using `sbt`: + +1. Add an [inline dependency](https://www.scala-sbt.org/1.x/docs/Library-Management.html#Dependencies) to `build.sbt`: + + ```scala + libraryDependencies += "com.mycompany.mydepartment" % "my-project" % "8.4" + ``` + +1. In your project, run the following: + + ```shell + sbt update + ``` + +::EndTabs + ## Helpful hints ### Publishing a package with the same name or version @@ -232,22 +497,19 @@ to [Maven Central](https://search.maven.org/). When the feature flag is enabled, administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md). -There are many ways to configure your Maven project so that it requests packages -in Maven Central from GitLab. Maven repositories are queried in a -[specific order](https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order). -By default, maven-central is usually checked first through the -[Super POM](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM), so -GitLab needs to be configured to be queried before maven-central. - -[Using GitLab as a mirror of the central proxy](#setting-gitlab-as-a-mirror-for-the-central-proxy) is one -way to force GitLab to be queried in place of maven-central. - Maven forwarding is restricted to only the project level and group level [endpoints](#naming-convention). The instance level endpoint has naming restrictions that prevent it from being used for packages that don't follow that convention and also introduces too much security risk for supply-chain style attacks. -#### Setting GitLab as a mirror for the central proxy +#### Additional configuration for `mvn` + +When using `mvn`, there are many ways to configure your Maven project so that it requests packages +in Maven Central from GitLab. Maven repositories are queried in a +[specific order](https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order). +By default, Maven Central is usually checked first through the +[Super POM](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM), so +GitLab needs to be configured to be queried before maven-central. To ensure all package requests are sent to GitLab instead of Maven Central, you can override Maven Central as the central repository by adding a `<mirror>` @@ -284,9 +546,11 @@ section to your `settings.xml`: After you have configured your repository to use the Package Repository for Maven, you can configure GitLab CI/CD to build new packages automatically. -### Create Maven packages with GitLab CI/CD using Maven +::Tabs + +:::TabTitle `mvn` -You can create a new package each time the `main` branch is updated. +You can create a new package each time the default branch is updated. 1. Create a `ci_settings.xml` file that serves as Maven's `settings.xml` file. @@ -342,8 +606,8 @@ You can create a new package each time the `main` branch is updated. image: maven:3.6-jdk-11 script: - 'mvn deploy -s ci_settings.xml' - only: - - main + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH ``` 1. Push those files to your repository. @@ -354,6 +618,30 @@ user's home location. In this example: - The user is `root`, because the job runs in a Docker container. - Maven uses the configured CI/CD variables. +:::TabTitle `gradle` + +You can create a package each time the default branch +is updated. + +1. Authenticate with [a CI job token in Gradle](#edit-the-client-configuration). + +1. Add a `deploy` job to your `.gitlab-ci.yml` file: + + ```yaml + deploy: + image: gradle:6.5-jdk11 + script: + - 'gradle publish' + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + ``` + +1. Commit files to your repository. + +When the pipeline is successful, the Maven package is created. + +::EndTabs + ### Version validation The version string is validated by using the following regex. @@ -394,27 +682,44 @@ that you can use when performing tasks with GitLab CI/CD. ### Supported CLI commands -The GitLab Maven repository supports the following Maven CLI commands: +The GitLab Maven repository supports the following CLI commands: + +::Tabs + +:::TabTitle `mvn` - `mvn deploy`: Publish your package to the Package Registry. - `mvn install`: Install packages specified in your Maven project. - `mvn dependency:get`: Install a specific package. +:::TabTitle `gradle` + +- `gradle publish`: Publish your package to the Package Registry. +- `gradle install`: Install packages specified in your Gradle project. + +::EndTabs + ## Troubleshooting -To improve performance, Maven caches files related to a package. If you encounter issues, clear +To improve performance, clients cache files related to a package. If you encounter issues, clear the cache with these commands: +::Tabs + +:::TabTitle `mvn` + ```shell rm -rf ~/.m2/repository ``` -If you're using Gradle, run this command to clear the cache: +:::TabTitle `gradle` ```shell rm -rf ~/.gradle/caches # Or replace ~/.gradle with your custom GRADLE_USER_HOME ``` +::EndTabs + ### Review network trace logs If you are having issues with the Maven Repository, you may want to review network trace logs. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 11e3d0e5131..52fdda0d523 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -23,6 +23,8 @@ You need an token to publish a package. There are different tokens available dep Create a token and save it to use later in the process. +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ### Naming convention Depending on how the package is installed, you may need to adhere to the naming convention. @@ -122,21 +124,50 @@ You can install a package from a GitLab project or instance: - **Instance-level**: Use when you have many npm packages in different GitLab groups or in their own namespace. - **Project-level**: Use when you have few npm packages and they are not in the same GitLab group. -### Install from the instance level +### Authenticate to the Package Registry -WARNING: -To install a package from the instance level, the package must have been published following the scoped [naming convention](#naming-convention). +You must authenticate to the Package Registry to install a package from a private project. +No authentication is needed if the project is public. -1. Authenticate to the Package Registry +To authenticate with `npm`: - If you would like to install a package from a private project, you would have to authenticate to the Package Registry. Skip this step if the project is not private. +```shell +npm config set -- //your_domain_name/:_authToken=your_token +``` - ```shell - npm config set -- //your_domain_name/api/v4/packages/npm/:_authToken=your_token - ``` +With npm version 7 or earlier, use the full URL to the endpoint. - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +If you're installing: + +- From the instance level: + + ```shell + npm config set -- //your_domain_name/api/v4/packages/npm/:_authToken=your_token + ``` + + From the project level: + + ```shell + npm config set -- //your_domain_name/api/v4/projects/your_project_id/packages/npm/:_authToken=your_token + ``` + +In these examples: + +- Replace `your_domain_name` with your domain name, for example, `gitlab.com`. +- Replace `your_project_id` is your project ID, found on the project's home page. +- Replace `your_token` with a deploy token, group access token, project access token, or personal access token. + +NOTE: +Starting with npm version 8, you can [use a URI fragment instead of a full URL](https://docs.npmjs.com/cli/v8/configuring-npm/npmrc?v=true#auth-related-configuration) +in the `_authToken` parameter. However, [group-level endpoints](https://gitlab.com/gitlab-org/gitlab/-/issues/299834) +are not supported. + +### Install from the instance level + +WARNING: +To install a package from the instance level, the package must have been published following the scoped [naming convention](#naming-convention). + +1. [Authenticate to the Package Registry](#authenticate-to-the-package-registry). 1. Set the registry @@ -156,17 +187,7 @@ To install a package from the instance level, the package must have been publish ### Install from the project level -1. Authenticate to the Package Registry - - If you would like to install a package from a private project, you would have to authenticate to the Package Registry. Skip this step if the project is not private. - - ```shell - npm config set -- //your_domain_name/api/v4/projects/your_project_id/packages/npm/:_authToken=your_token - ``` - - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_project_id` is your project ID, found on the project's home page. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +1. [Authenticate to the Package Registry](#authenticate-to-the-package-registry). 1. Set the registry @@ -184,6 +205,39 @@ To install a package from the instance level, the package must have been publish npm install @scope/my-package ``` +## Deprecate a package + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/396763) in GitLab 16.0. + +You can deprecate a package so that a deprecation warning displays when the package is fetched. + +Pre-requisites: + +- The same [permissions](../../permissions.md) as deleting a package. +- [Authenticated to the package registry](#authentication-to-the-package-registry). + +From the command line, run: + +```shell +npm deprecate @scope/package "Deprecation message" +``` + +The CLI also accepts version ranges for `@scope/package`. For example: + +```shell +npm deprecate @scope/package "All package versions are deprecated" +npm deprecate @scope/package@1.0.1 "Only version 1.0.1 is deprecated" +npm deprecate @scope/package@"< 1.0.5" "All package versions less than 1.0.5 are deprecated" +``` + +### Remove deprecation warning + +To remove a package's deprecation warning, specify `""` (an empty string) for the message. For example: + +```shell +npm deprecate @scope/package "" +``` + ## Helpful hints ### Package forwarding to npmjs.com @@ -244,8 +298,19 @@ npm dist-tag rm @scope/package@version my-tag # Delete a tag from the package npm install @scope/package@my-tag # Install a specific tag ``` -You cannot use your `CI_JOB_TOKEN` or deploy token with the `npm dist-tag` commands. -View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/258835) for details. +#### From CI/CD + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258835) in GitLab 15.10. + +You can use a [`CI_JOB_TOKEN`](../../../ci/jobs/ci_job_token.md) or [deploy token](../../project/deploy_tokens/index.md) +to run `npm dist-tag` commands in a GitLab CI/CD job. For example: + +```yaml +npm-deploy-job: + script: + - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc + - npm dist-tag add @scope/package@version my-tag +``` Due to a bug in npm 6.9.0, deleting distribution tags fails. Make sure your npm version is 6.9.1 or later. @@ -261,24 +326,35 @@ The GitLab npm repository supports the following commands for the npm CLI (`npm` - `npm dist-tag rm`: Delete a dist-tag. - `npm ci`: Install npm packages directly from your `package-lock.json` file. - `npm view`: Show package metadata. +- `npm pack`: Create a tarball from a package. +- `npm deprecate`: Deprecate a version of a package. ## Troubleshooting ### `404 Not Found` errors are happening on `npm install` or `yarn` -Using `CI_JOB_TOKEN` to install npm packages with dependencies in another project gives you 404 Not Found errors. A fix for this problem is proposed in [issue 352962](https://gitlab.com/gitlab-org/gitlab/-/issues/352962). +Using `CI_JOB_TOKEN` to install npm packages with dependencies in another project gives you 404 Not Found errors. You need to authenticate with a token that has access to the package and all its dependencies. -As a workaround, you can: +If the package and its dependencies are in separate projects but in the same group, you can use a +[group deploy token](../../project/deploy_tokens/index.md#create-a-deploy-token): -1. Create a [personal access token](../../profile/personal_access_tokens.md). -1. Authenticate at both the instance level and project level for each package: +```ini +//gitlab.example.com/api/v4/packages/npm/:_authToken=<group-token> +@group-scope:registry=https://gitlab.example.com/api/v4/packages/npm/ +``` - ```ini - @foo:registry=https://gitlab.example.com/api/v4/packages/npm/ - //gitlab.example.com/api/v4/packages/npm/:_authToken=${MY_TOKEN} - //gitlab.example.com/api/v4/projects/<your_project_id_a>/packages/npm/:_authToken=${MY_TOKEN} - //gitlab.example.com/api/v4/projects/<your_project_id_b>/packages/npm/:_authToken=${MY_TOKEN} - ``` +If the package and its dependencies are spread across multiple groups, you can use a [personal access token](../../profile/personal_access_tokens.md) +from a user that has access to all the groups or individual projects: + +```ini +//gitlab.example.com/api/v4/packages/npm/:_authToken=<personal-access-token> +@group-1:registry=https://gitlab.example.com/api/v4/packages/npm/ +@group-2:registry=https://gitlab.example.com/api/v4/packages/npm/ +``` + +WARNING: +Personal access tokens must be treated carefully. Read our [token security considerations](../../../security/token_overview.md#security-considerations) +for guidance on managing personal access tokens (for example, setting a short expiry and using minimal scopes). ### `npm publish` targets default npm registry (`registry.npmjs.org`) diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 6710c147c87..c97ce9ec593 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -39,6 +39,8 @@ Some features such as [publishing](#publish-a-nuget-package) a package are only When asking for versions of a given NuGet package name, the GitLab Package Registry returns a maximum of 300 most recent versions. +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + WARNING: Because of how NuGet handles credentials, the Package Registry rejects anonymous requests on the group-level endpoint. To work around this limitation, set up [authentication](#add-the-package-registry-as-a-source-for-nuget-packages). @@ -472,4 +474,4 @@ nuget locals all -clear ### `Error publishing` or `Invalid Package: Failed metadata extraction error` messages when trying to publish NuGet packages in a Docker-based GitLab installation -Webhook requests to local network addresses are blocked to prevent the exploitation of internal web services. If you get `Error publishing` or `Invalid Package` messages when you try to publish NuGet packages, change your network settings to [allow webhook and service requests to the local network](../../../security/webhooks.md#allow-webhook-and-service-requests-to-local-network). +Webhook requests to local network addresses are blocked to prevent exploitation of internal web services. If you get `Error publishing` or `Invalid Package` messages when you try to publish NuGet packages, change your network settings to [allow webhook and integration requests to the local network](../../../security/webhooks.md#allow-requests-to-the-local-network-from-webhooks-and-integrations). diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index cd60a229479..91186e6c159 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -71,9 +71,13 @@ NOTE: If you have not activated the "Package registry" feature for your project at **Settings > General > Visibility, project features, permissions**, you receive a 403 Forbidden response. Accessing package registry via deploy token is not available when external authorization is enabled. -## Use GitLab CI/CD to build packages +## Use GitLab CI/CD + +You can use [GitLab CI/CD](../../../ci/index.md) to build or import packages into +a package registry. + +### To build packages -You can use [GitLab CI/CD](../../../ci/index.md) to build packages. For Maven, NuGet, npm, Conan, Helm, and PyPI packages, and Composer dependencies, you can authenticate with GitLab by using the `CI_JOB_TOKEN`. @@ -97,6 +101,16 @@ when you view the package details: You can view which pipeline published the package, and the commit and user who triggered it. However, the history is limited to five updates of a given package. +### To import packages + +If you already have packages built in a different registry, you can import them +into your GitLab package registry with the [Packages Importer](https://gitlab.com/gitlab-org/ci-cd/package-stage/pkgs_importer) + +The Packages Importer runs a CI/CD pipeline that [can import these package types](https://gitlab.com/gitlab-org/ci-cd/package-stage/pkgs_importer#formats-supported): + +- NPM +- NuGet + ## Reduce storage usage For information on reducing your storage use for the Package Registry, see @@ -129,17 +143,17 @@ your project's settings. For example, if you have a public project and set the r to **Only Project Members**, the Package Registry is then public. Disabling the Package Registry disables all Package Registry operations. -| Project visibility | Action | [Role](../../permissions.md#roles) required | +| Project visibility | Action | Minimum [role](../../permissions.md#roles) required | |--------------------|-----------------------|---------------------------------------------------------| | Public | View Package Registry | `n/a`, everyone on the internet can perform this action | -| Public | Publish a package | Developer or higher | +| Public | Publish a package | Developer | | Public | Pull a package | `n/a`, everyone on the internet can perform this action | -| Internal | View Package Registry | Guest or higher | -| Internal | Publish a package | Developer or higher | -| Internal | Pull a package | Guest or higher(1) | -| Private | View Package Registry | Reporter or higher | -| Private | Publish a package | Developer or higher | -| Private | Pull a package | Reporter or higher(1) | +| Internal | View Package Registry | Guest | +| Internal | Publish a package | Developer | +| Internal | Pull a package | Guest (1) | +| Private | View Package Registry | Reporter | +| Private | Publish a package | Developer | +| Private | Pull a package | Reporter (1) | ### Allow anyone to pull from Package Registry diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md index 673196ebad5..020cad5ac14 100644 --- a/doc/user/packages/package_registry/reduce_package_registry_storage.md +++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md @@ -35,6 +35,9 @@ To delete a package in the UI, from your group or project: The package is permanently deleted. +If [request forwarding](supported_functionality.md#forwarding-requests) is enabled, +deleting a package can introduce a [dependency confusion risk](supported_functionality.md#deleting-packages). + ## Delete assets associated with a package To delete package assets, you must have suitable [permissions](../../permissions.md). diff --git a/doc/user/packages/package_registry/supported_functionality.md b/doc/user/packages/package_registry/supported_functionality.md index e56ae88872a..ca174c43565 100644 --- a/doc/user/packages/package_registry/supported_functionality.md +++ b/doc/user/packages/package_registry/supported_functionality.md @@ -13,77 +13,102 @@ and pulling packages, request forwarding, managing duplicates, and authenticatio Packages can be published to your project, group, or instance. -| Package type | Project | Group | Instance | -|-----------------------------------------------------|---------|-------|----------| -| [Maven](../maven_repository/index.md) | Y | N | N | -| [npm](../npm_registry/index.md) | Y | N | N | -| [NuGet](../nuget_repository/index.md) | Y | N | N | -| [PyPI](../pypi_repository/index.md) | Y | N | N | -| [Generic packages](../generic_packages/index.md) | Y | N | N | -| [Terraform](../terraform_module_registry/index.md) | Y | N | N | -| [Composer](../composer_repository/index.md) | N | Y | N | -| [Conan](../conan_repository/index.md) | Y | N | N | -| [Helm](../helm_repository/index.md) | Y | N | N | -| [Debian](../debian_repository/index.md) | Y | N | N | -| [Go](../go_proxy/index.md) | Y | N | Y | -| [Ruby gems](../rubygems_registry/index.md) | Y | N | N | +| Package type | Project | Group | Instance | +|-------------------------------------------------------|---------|-------|----------| +| [Maven (with `mvn`)](../maven_repository/index.md) | Y | N | N | +| [Maven (with `gradle`)](../maven_repository/index.md) | Y | N | N | +| [Maven (with `sbt`)](../maven_repository/index.md) | N | N | N | +| [npm](../npm_registry/index.md) | Y | N | N | +| [NuGet](../nuget_repository/index.md) | Y | N | N | +| [PyPI](../pypi_repository/index.md) | Y | N | N | +| [Generic packages](../generic_packages/index.md) | Y | N | N | +| [Terraform](../terraform_module_registry/index.md) | Y | N | N | +| [Composer](../composer_repository/index.md) | N | Y | N | +| [Conan](../conan_repository/index.md) | Y | N | Y | +| [Helm](../helm_repository/index.md) | Y | N | N | +| [Debian](../debian_repository/index.md) | Y | N | N | +| [Go](../go_proxy/index.md) | Y | N | N | +| [Ruby gems](../rubygems_registry/index.md) | Y | N | N | ## Pulling packages **(FREE)** Packages can be pulled from your project, group, or instance. -| Package type | Project | Group | Instance | -|-----------------------------------------------------|---------|-------|----------| -| [Maven](../maven_repository/index.md) | Y | Y | Y | -| [npm](../npm_registry/index.md) | Y | Y | Y | -| [NuGet](../nuget_repository/index.md) | Y | Y | N | -| [PyPI](../pypi_repository/index.md) | Y | Y | N | -| [Generic packages](../generic_packages/index.md) | Y | N | N | -| [Terraform](../terraform_module_registry/index.md) | N | Y | N | -| [Composer](../composer_repository/index.md) | Y | Y | N | -| [Conan](../conan_repository/index.md) | Y | N | Y | -| [Helm](../helm_repository/index.md) | Y | N | N | -| [Debian](../debian_repository/index.md) | Y | N | N | -| [Go](../go_proxy/index.md) | Y | N | Y | -| [Ruby gems](../rubygems_registry/index.md) | Y | N | N | +| Package type | Project | Group | Instance | +|-------------------------------------------------------|---------|-------|----------| +| [Maven (with `mvn`)](../maven_repository/index.md) | Y | Y | Y | +| [Maven (with `gradle`)](../maven_repository/index.md) | Y | Y | Y | +| [Maven (with `sbt`)](../maven_repository/index.md) | Y | Y | Y | +| [npm](../npm_registry/index.md) | Y | N | Y | +| [NuGet](../nuget_repository/index.md) | Y | Y | N | +| [PyPI](../pypi_repository/index.md) | Y | Y | N | +| [Generic packages](../generic_packages/index.md) | Y | N | N | +| [Terraform](../terraform_module_registry/index.md) | N | Y | N | +| [Composer](../composer_repository/index.md) | Y | Y | N | +| [Conan](../conan_repository/index.md) | Y | N | Y | +| [Helm](../helm_repository/index.md) | Y | N | N | +| [Debian](../debian_repository/index.md) | Y | N | N | +| [Go](../go_proxy/index.md) | Y | N | Y | +| [Ruby gems](../rubygems_registry/index.md) | Y | N | N | ## Forwarding requests **(PREMIUM)** Requests for packages not found in your GitLab project are forwarded to the public registry. For example, Maven Central, npmjs, or PyPI. -| Package type | Supports request forwarding | -|-----------------------------------------------------|-----------------------------| -| [Maven](../maven_repository/index.md) | Y | -| [npm](../npm_registry/index.md) | Y | -| [NuGet](../nuget_repository/index.md) | N | -| [PyPI](../pypi_repository/index.md) | Y | -| [Generic packages](../generic_packages/index.md) | N | -| [Terraform](../terraform_module_registry/index.md) | N | -| [Composer](../composer_repository/index.md) | N | -| [Conan](../conan_repository/index.md) | N | -| [Helm](../helm_repository/index.md) | N | -| [Debian](../debian_repository/index.md) | N | -| [Go](../go_proxy/index.md) | N | -| [Ruby gems](../rubygems_registry/index.md) | N | +| Package type | Supports request forwarding | +|-------------------------------------------------------|-----------------------------| +| [Maven (with `mvn`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | +| [Maven (with `gradle`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | +| [Maven (with `sbt`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | +| [npm](../npm_registry/index.md) | [Yes](../../admin_area/settings/continuous_integration.md#npm-forwarding) | +| [NuGet](../nuget_repository/index.md) | N | +| [PyPI](../pypi_repository/index.md) | [Yes](../../admin_area/settings/continuous_integration.md#pypi-forwarding) | +| [Generic packages](../generic_packages/index.md) | N | +| [Terraform](../terraform_module_registry/index.md) | N | +| [Composer](../composer_repository/index.md) | N | +| [Conan](../conan_repository/index.md) | N | +| [Helm](../helm_repository/index.md) | N | +| [Debian](../debian_repository/index.md) | N | +| [Go](../go_proxy/index.md) | N | +| [Ruby gems](../rubygems_registry/index.md) | N | + +### Deleting packages + +When package requests are forwarded to a public registry, deleting packages can +be a [dependency confusion vulnerability](https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610). + +If a system tries to pull a deleted package, the request is forwarded to the public +registry. If a package with the same name and version is found in the public registry, that package +is pulled instead. There is a risk that the package pulled from the registry might not be +what is expected, and could even be malicious. + +To reduce the associated security risks, before deleting a package you can: + +- Verify the package is not being actively used. +- Disable request forwarding: + - Instance administrators can disable forwarding in the [**Continuous Integration** section](../../admin_area/settings/continuous_integration.md#package-registry-configuration) of the Admin Area. + - Group owners can disable forwarding in the **Packages and Registries** section of the group settings. ## Allow or prevent duplicates **(FREE)** By default, the GitLab package registry either allows or prevents duplicates based on the default of that specific package manager format. -| Package type | Duplicates allowed? | -|-----------------------------------------------------|---------------------| -| [Maven](../maven_repository/index.md) | Y (configurable) | -| [npm](../npm_registry/index.md) | N | -| [NuGet](../nuget_repository/index.md) | Y | -| [PyPI](../pypi_repository/index.md) | N | -| [Generic packages](../generic_packages/index.md) | Y (configurable) | -| [Terraform](../terraform_module_registry/index.md) | N | -| [Composer](../composer_repository/index.md) | N | -| [Conan](../conan_repository/index.md) | N | -| [Helm](../helm_repository/index.md) | Y | -| [Debian](../debian_repository/index.md) | Y | -| [Go](../go_proxy/index.md) | N | -| [Ruby gems](../rubygems_registry/index.md) | Y | +| Package type | Duplicates allowed? | +|-------------------------------------------------------|---------------------| +| [Maven (with `mvn`)](../maven_repository/index.md) | Y (configurable) | +| [Maven (with `gradle`)](../maven_repository/index.md) | Y (configurable) | +| [Maven (with `sbt`)](../maven_repository/index.md) | Y (configurable) | +| [npm](../npm_registry/index.md) | N | +| [NuGet](../nuget_repository/index.md) | Y | +| [PyPI](../pypi_repository/index.md) | N | +| [Generic packages](../generic_packages/index.md) | Y (configurable) | +| [Terraform](../terraform_module_registry/index.md) | N | +| [Composer](../composer_repository/index.md) | N | +| [Conan](../conan_repository/index.md) | N | +| [Helm](../helm_repository/index.md) | Y | +| [Debian](../debian_repository/index.md) | Y | +| [Go](../go_proxy/index.md) | N | +| [Ruby gems](../rubygems_registry/index.md) | Y | ## Authentication tokens **(FREE)** @@ -91,39 +116,45 @@ GitLab tokens are used to authenticate with the GitLab Package Registry. The following tokens are supported: -| Package type | Supported tokens | -|-----------------------------------------------------|------------------------------------------------------------------------| -| [Maven](../maven_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [npm](../npm_registry/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [NuGet](../nuget_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [PyPI](../pypi_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [Generic packages](../generic_packages/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [Terraform](../terraform_module_registry/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [Composer](../composer_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | -| [Conan](../conan_repository/index.md) | Personal access, job tokens, project access | -| [Helm](../helm_repository/index.md) | Personal access, job tokens, deploy (project or group) | -| [Debian](../debian_repository/index.md) | Personal access, job tokens, deploy (project or group) | -| [Go](../go_proxy/index.md) | Personal access, job tokens, project access | -| [Ruby gems](../rubygems_registry/index.md) | Personal access, job tokens, deploy (project or group) | +| Package type | Supported tokens | +|-------------------------------------------------------|------------------------------------------------------------------------| +| [Maven (with `mvn`)](../maven_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Maven (with `gradle`)](../maven_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Maven (with `sbt`)](../maven_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [npm](../npm_registry/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [NuGet](../nuget_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [PyPI](../pypi_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Generic packages](../generic_packages/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Terraform](../terraform_module_registry/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Composer](../composer_repository/index.md) | Personal access, job tokens, deploy (project or group), project access | +| [Conan](../conan_repository/index.md) | Personal access, job tokens, project access | +| [Helm](../helm_repository/index.md) | Personal access, job tokens, deploy (project or group) | +| [Debian](../debian_repository/index.md) | Personal access, job tokens, deploy (project or group) | +| [Go](../go_proxy/index.md) | Personal access, job tokens, project access | +| [Ruby gems](../rubygems_registry/index.md) | Personal access, job tokens, deploy (project or group) | ## Authentication protocols **(FREE)** The following authentication protocols are supported: -| Package type | Supported auth protocols | -|-----------------------------------------------------|--------------------------| -| [Maven](../maven_repository/index.md) | Headers | -| [npm](../npm_registry/index.md) | OAuth | -| [NuGet](../nuget_repository/index.md) | Basic auth | -| [PyPI](../pypi_repository/index.md) | Basic auth | -| [Generic packages](../generic_packages/index.md) | Basic auth | -| [Terraform](../terraform_module_registry/index.md) | Token | -| [Composer](../composer_repository/index.md) | OAuth | -| [Conan](../conan_repository/index.md) | OAuth, Basic auth | -| [Helm](../helm_repository/index.md) | Basic auth | -| [Debian](../debian_repository/index.md) | Basic auth | -| [Go](../go_proxy/index.md) | Basic auth | -| [Ruby gems](../rubygems_registry/index.md) | Token | +| Package type | Supported auth protocols | +|-------------------------------------------------------|-------------------------------------------------------------| +| [Maven (with `mvn`)](../maven_repository/index.md) | Headers, Basic auth ([pulling](#pulling-packages) only) (1) | +| [Maven (with `gradle`)](../maven_repository/index.md) | Headers, Basic auth ([pulling](#pulling-packages) only) (1) | +| [Maven (with `sbt`)](../maven_repository/index.md) | Basic auth (1) | +| [npm](../npm_registry/index.md) | OAuth | +| [NuGet](../nuget_repository/index.md) | Basic auth | +| [PyPI](../pypi_repository/index.md) | Basic auth | +| [Generic packages](../generic_packages/index.md) | Basic auth | +| [Terraform](../terraform_module_registry/index.md) | Token | +| [Composer](../composer_repository/index.md) | OAuth | +| [Conan](../conan_repository/index.md) | OAuth, Basic auth | +| [Helm](../helm_repository/index.md) | Basic auth | +| [Debian](../debian_repository/index.md) | Basic auth | +| [Go](../go_proxy/index.md) | Basic auth | +| [Ruby gems](../rubygems_registry/index.md) | Token | + +1. Basic authentication for Maven packages [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212854) in GitLab 16.0. ## Supported hash types **(FREE)** @@ -131,16 +162,18 @@ Hash values are used to ensure you are using the correct package. You can view t The Package Registry supports the following hash types: -| Package type | Supported hashes | -|--------------------------------------------------|----------------------------------| -| [Maven](../maven_repository/index.md) | MD5, SHA1 | -| [npm](../npm_registry/index.md) | SHA1 | -| [NuGet](../nuget_repository/index.md) | not applicable | -| [PyPI](../pypi_repository/index.md) | MD5, SHA256 | -| [Generic packages](../generic_packages/index.md) | SHA256 | -| [Composer](../composer_repository/index.md) | not applicable | -| [Conan](../conan_repository/index.md) | MD5, SHA1 | -| [Helm](../helm_repository/index.md) | not applicable | -| [Debian](../debian_repository/index.md) | MD5, SHA1, SHA256 | -| [Go](../go_proxy/index.md) | MD5, SHA1, SHA256 | -| [Ruby gems](../rubygems_registry/index.md) | MD5, SHA1, SHA256 (gemspec only) | +| Package type | Supported hashes | +|-------------------------------------------------------|----------------------------------| +| [Maven (with `mvn`)](../maven_repository/index.md) | MD5, SHA1 | +| [Maven (with `gradle`)](../maven_repository/index.md) | MD5, SHA1 | +| [Maven (with `sbt`)](../maven_repository/index.md) | MD5, SHA1 | +| [npm](../npm_registry/index.md) | SHA1 | +| [NuGet](../nuget_repository/index.md) | not applicable | +| [PyPI](../pypi_repository/index.md) | MD5, SHA256 | +| [Generic packages](../generic_packages/index.md) | SHA256 | +| [Composer](../composer_repository/index.md) | not applicable | +| [Conan](../conan_repository/index.md) | MD5, SHA1 | +| [Helm](../helm_repository/index.md) | not applicable | +| [Debian](../debian_repository/index.md) | MD5, SHA1, SHA256 | +| [Go](../go_proxy/index.md) | MD5, SHA1, SHA256 | +| [Ruby gems](../rubygems_registry/index.md) | MD5, SHA1, SHA256 (gemspec only) | diff --git a/doc/user/packages/package_registry/supported_hash_types.md b/doc/user/packages/package_registry/supported_hash_types.md deleted file mode 100644 index d39b8d60c93..00000000000 --- a/doc/user/packages/package_registry/supported_hash_types.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'supported_functionality.md' -remove_date: '2023-04-22' ---- - -This document was moved to [another location](supported_functionality.md). - -<!-- This redirect file can be deleted after <2023-04-22>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/packages/package_registry/supported_package_managers.md b/doc/user/packages/package_registry/supported_package_managers.md index 75b8c95a0fa..cd2b288094b 100644 --- a/doc/user/packages/package_registry/supported_package_managers.md +++ b/doc/user/packages/package_registry/supported_package_managers.md @@ -11,24 +11,20 @@ Not all package manager formats are ready for production use. The Package Registry supports the following package manager types: -| Package type | GitLab version | Status | -| ------------------------------------------------ | -------------- | ---------------------------------------------------------- | -| [Maven](../maven_repository/index.md) | 11.3+ | GA | -| [npm](../npm_registry/index.md) | 11.7+ | GA | -| [NuGet](../nuget_repository/index.md) | 12.8+ | GA | -| [PyPI](../pypi_repository/index.md) | 12.10+ | GA | -| [Generic packages](../generic_packages/index.md) | 13.5+ | GA | -| [Composer](../composer_repository/index.md) | 13.2+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6817) | -| [Conan](../conan_repository/index.md) | 12.6+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6816) | -| [Helm](../helm_repository/index.md) | 14.1+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6366) | -| [Debian](../debian_repository/index.md) | 14.2+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/6057) | -| [Go](../go_proxy/index.md) | 13.1+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/3043) | -| [Ruby gems](../rubygems_registry/index.md) | 13.10+ | [Alpha](https://gitlab.com/groups/gitlab-org/-/epics/3200) | +| Package type | GitLab version | Status | +| ------------------------------------------------ | -------------- | --------------------------------------------------------------- | +| [Maven](../maven_repository/index.md) | 11.3+ | GA | +| [npm](../npm_registry/index.md) | 11.7+ | GA | +| [NuGet](../nuget_repository/index.md) | 12.8+ | GA | +| [PyPI](../pypi_repository/index.md) | 12.10+ | GA | +| [Generic packages](../generic_packages/index.md) | 13.5+ | GA | +| [Composer](../composer_repository/index.md) | 13.2+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6817) | +| [Helm](../helm_repository/index.md) | 14.1+ | [Beta](https://gitlab.com/groups/gitlab-org/-/epics/6366) | +| [Conan](../conan_repository/index.md) | 12.6+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/6816) | +| [Debian](../debian_repository/index.md) | 14.2+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/6057) | +| [Go](../go_proxy/index.md) | 13.1+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/3043) | +| [Ruby gems](../rubygems_registry/index.md) | 13.10+ | [Experiment](https://gitlab.com/groups/gitlab-org/-/epics/3200) | -[Status](../../../policy/alpha-beta-support.md): - -- Alpha: behind a feature flag and not officially supported. -- Beta: several known issues that may prevent expected use. -- GA (Generally Available): ready for production use at any scale. +[View what each status means](../../../policy/alpha-beta-support.md). You can also use the [API](../../../api/packages.md) to administer the Package Registry. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index e5b7f06f6ca..d23966f26fe 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -34,6 +34,8 @@ To do this, you can use: `read_package_registry`, `write_package_registry`, or both. - A [CI job token](#authenticate-with-a-ci-job-token). +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. + ### Authenticate with a personal access token To authenticate with a personal access token, edit the `~/.pypirc` file and add: diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index 66a42f398b0..d7f33dd9e79 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -4,30 +4,53 @@ group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Terraform module registry **(FREE)** +# Terraform Module Registry **(FREE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3221) in GitLab 14.0. +> - Infrastructure registry and Terraform Module Registry [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/404075) into a single Terraform Module Registry feature in GitLab 15.11. -Publish Terraform modules in your project's Infrastructure Registry, then reference them using GitLab -as a Terraform module registry. +With the Terraform Module Registry, you can use GitLab projects as a +private registry for terraform modules. You can create and publish +modules with GitLab CI/CD, which can then be consumed from other private +projects. -## Authenticate to the Terraform module registry +## View Terraform modules -To authenticate to the Terraform module registry, you need either: +To view Terraform modules in your project: + +1. Go to the project. +1. On the left sidebar, select **Packages and registries > Terraform modules**. + +You can search, sort, and filter modules on this page. + +For information on how to create and upload a package, view the GitLab +documentation for your package type: + +- [Terraform modules](../terraform_module_registry/index.md) + +## Authenticate to the Terraform Module Registry + +To authenticate to the Terraform Module Registry, you need either: - A [personal access token](../../../api/rest/index.md#personalprojectgroup-access-tokens) with at least `read_api` rights. - A [CI/CD job token](../../../ci/jobs/ci_job_token.md). -## Publish a Terraform Module +Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future. -When you publish a Terraform Module, if it does not exist, it is created. +## Publish a Terraform module + +When you publish a Terraform module, if it does not exist, it is created. + +### Using the API + +You can publish Terraform modules by using the [Terraform Module Registry API](../../../api/packages/terraform-modules.md). Prerequisites: -- The package name and version [must be unique in the top-level namespace](../infrastructure_registry/index.md#how-module-resolution-works). +- The package name and version [must be unique in the top-level namespace](#how-module-resolution-works). - Your project and group names must not include a dot (`.`). For example, `source = "gitlab.example.com/my.group/project.name"`. - You must [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. -- The name of a module [must be unique within the scope of its group](../infrastructure_registry/index.md#how-module-resolution-works), otherwise an +- The name of a module [must be unique in the scope of its group](#how-module-resolution-works), otherwise an [error occurs](#troubleshooting). ```plaintext @@ -37,9 +60,9 @@ PUT /projects/:id/packages/terraform/modules/:module-name/:module-system/:module | Attribute | Type | Required | Description | | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/rest/index.md#namespaced-path-encoding). | -| `module-name` | string | yes | The package name. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The package name can't exceed 64 characters. -| `module-system` | string | yes | The package system. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The package system can't exceed 64 characters. More information can be found in the [Terraform Module Registry Protocol documentation](https://www.terraform.io/internals/module-registry-protocol). -| `module-version` | string | yes | The package version. It must be valid according to the [Semantic Versioning Specification](https://semver.org/). +| `module-name` | string | yes | The module name. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The module name can't exceed 64 characters. +| `module-system` | string | yes | The module system. **Supported syntax**: One to 64 ASCII characters, including lowercase letters (a-z) and digits (0-9). The module system can't exceed 64 characters. More information can be found in the [Terraform Module Registry protocol documentation](https://www.terraform.io/internals/module-registry-protocol). +| `module-version` | string | yes | The module version. It must be valid according to the [semantic versioning specification](https://semver.org/). Provide the file content in the request body. @@ -55,14 +78,6 @@ curl --fail-with-body --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/terraform/modules/my-module/my-system/0.0.1/file" ``` -Example response: - -```json -{ - "message":"201 Created" -} -``` - Example request using a deploy token: ```shell @@ -79,41 +94,13 @@ Example response: } ``` -## Reference a Terraform Module +### Using a CI/CD template (recommended) -Prerequisites: - -- You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. - -Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: - -```plaintext -credentials "gitlab.com" { - token = "<TOKEN>" -} -``` - -Where `gitlab.com` can be replaced with the hostname of your self-managed GitLab instance. - -You can then refer to your Terraform Module from a downstream Terraform project: - -```plaintext -module "<module>" { - source = "gitlab.com/<namespace>/<module-name>/<module-system>" -} -``` - -Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the Terraform module registry. - -## Publish a Terraform module by using CI/CD - -> CI/CD template [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110493) in GitLab 15.9. - -### Use a CI/CD template (recommended) +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110493) in GitLab 15.9. You can use the [`Terraform-Module.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform-Module.gitlab-ci.yml) or the advanced [`Terraform/Module-Base.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Module-Base.gitlab-ci.yml) -CI/CD template to publish a Terraform module to the GitLab Terraform Registry: +CI/CD template to publish a Terraform module to the GitLab terraform registry: ```yaml include: @@ -124,7 +111,7 @@ The pipeline contains the following jobs: - `fmt` - Validate the formatting of the Terraform module. - `kics-iac-sast` - Test the Terraform module for security issues. -- `deploy` - For tag pipelines only. Deploy the Terraform module to the GitLab Terraform Registry. +- `deploy` - For tag pipelines only. Deploy the Terraform module to the Terraform Module Registry. #### Pipeline variables @@ -137,7 +124,7 @@ You can configure the pipeline with the following variables: | `TERRAFORM_MODULE_SYSTEM` | `local` | The system or provider of your Terraform module targets. For example, `local`, `aws`, `google`. | | `TERRAFORM_MODULE_VERSION` | `${CI_COMMIT_TAG}` | The Terraform module version. You should follow the semantic versioning specification. | -### Deploy manually via CI/CD +### Using CI/CD manually To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. @@ -166,14 +153,97 @@ upload: - if: $CI_COMMIT_TAG ``` -To trigger this upload job, add a Git tag to your commit. Ensure the tag follows the [Semantic Versioning Specification](https://semver.org/) that Terraform requires. The `rules:if: $CI_COMMIT_TAG` ensures that only tagged commits to your repository trigger the module upload job. +To trigger this upload job, add a Git tag to your commit. Ensure the tag follows the [Semantic versioning specification](https://semver.org/) that Terraform requires. The `rules:if: $CI_COMMIT_TAG` ensures that only tagged commits to your repository trigger the module upload job. For other ways to control jobs in your CI/CD pipeline, refer to the [`.gitlab-ci.yml`](../../../ci/yaml/index.md) keyword reference. +## Reference a Terraform module + +Prerequisites: + +- You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. + +Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: + +```terraform +credentials "gitlab.com" { + token = "<TOKEN>" +} +``` + +Where `gitlab.com` can be replaced with the hostname of your self-managed GitLab instance. + +You can then refer to your Terraform module from a downstream Terraform project: + +```terraform +module "<module>" { + source = "gitlab.com/<namespace>/<module-name>/<module-system>" +} +``` + +Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the Terraform Module Registry. + +## Download a Terraform module + +To download a Terraform module: + +1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. Select the name of the module you want to download. +1. In the **Activity** section, select the name of the module you want to download. + +## How module resolution works + +When you upload a new module, GitLab generates a path for the module, for example, `https://gitlab.example.com/parent-group/my-infra-package`. + +- This path conforms with [the Terraform spec](https://www.terraform.io/internals/module-registry-protocol). +- The name of the path must be unique in the namespace. + +For projects in subgroups, GitLab checks that the module name does not already exist anywhere in the namespace, including all subgroups and the parent group. + +For example, if: + +- The project is `gitlab.example.com/parent-group/sub-group/my-project`. +- The Terraform module is `my-infra-package`. + +The project name must be unique in all projects in all groups under `parent-group`. + +## Delete a Terraform module + +You cannot edit a Terraform module after you publish it in the Terraform Module Registry. Instead, you +must delete and recreate it. + +To delete a module, you must have suitable [permissions](../../permissions.md). + +You can delete modules by using [the packages API](../../../api/packages.md#delete-a-project-package) or the UI. + +To delete a module in the UI, from your project: + +1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. Find the name of the package you want to delete. +1. Select **Delete**. + +The package is permanently deleted. + +## Disable the Terraform Module Registry + +The Terraform Module Registry is automatically enabled. + +For self-managed instances, a GitLab administrator can +[disable](../../../administration/packages/index.md) **Packages and registries**, +which removes this menu item from the sidebar. + +You can also remove the Terraform Module Registry for a specific project: + +1. In your project, go to **Settings > General**. +1. Expand the **Visibility, project features, permissions** section and toggle **Packages** off (in gray). +1. Select **Save changes**. + +To enable it back, follow the same steps above and toggle it on (in blue). + ## Example projects -For examples of the Terraform module registry, check the projects below: +For examples of the Terraform Module Registry, check the projects below: -- The [_GitLab local file_ project](https://gitlab.com/mattkasa/gitlab-local-file) creates a minimal Terraform module and uploads it into the Terraform module registry using GitLab CI/CD. +- The [_GitLab local file_ project](https://gitlab.com/mattkasa/gitlab-local-file) creates a minimal Terraform module and uploads it into the Terraform Module Registry using GitLab CI/CD. - The [_Terraform module test_ project](https://gitlab.com/mattkasa/terraform-module-test) uses the module from the previous example. ## Troubleshooting diff --git a/doc/user/packages/yarn_repository/index.md b/doc/user/packages/yarn_repository/index.md index 7e2f45019cd..c8f4985a518 100644 --- a/doc/user/packages/yarn_repository/index.md +++ b/doc/user/packages/yarn_repository/index.md @@ -6,220 +6,310 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Publish packages with Yarn -Publish npm packages in your project's Package Registry using Yarn. Then install the -packages whenever you need to use them as a dependency. +You can publish packages with [Yarn 1 (Classic)](https://classic.yarnpkg.com) and [Yarn 2+](https://yarnpkg.com). -Learn how to build a [yarn](../workflows/build_packages.md#yarn) package. +To find the Yarn version used in the deployment container, run `yarn --version` in the `script` block of the CI +script job block that is responsible for calling `yarn publish`**`. The Yarn version is shown in the pipeline output. -You can get started with Yarn 2 by following the [Yarn documentation](https://yarnpkg.com/getting-started/install/). +Learn how to build a [Yarn](../workflows/build_packages.md#yarn) package. + +You can use the Yarn documentation to get started with +[Yarn Classic](https://classic.yarnpkg.com/en/docs/getting-started) and +[Yarn 2+](https://yarnpkg.com/getting-started/). ## Publish to GitLab Package Registry +You can use Yarn to publish to the GitLab Package Registry. + ### Authentication to the Package Registry You need a token to publish a package. Different tokens are available depending on what you're trying to achieve. For more information, review the [guidance on tokens](../../../user/packages/package_registry/index.md#authenticate-with-the-registry). -- If your organization uses two-factor authentication (2FA), you must use a personal access token with the scope set to `api`. -- If you publish a package via CI/CD pipelines, you must use a CI job token. +- If your organization uses two-factor authentication (2FA), you must use a + personal access token with the scope set to `api`. +- If you publish a package via CI/CD pipelines, you can use a CI job token in + private runners or you can register a variable for shared runners. -Create a token and save it to use later in the process. +### Publish configuration -### Naming convention +To publish, set the following configuration in `.yarnrc.yml`. This file should be +located in the root directory of your package project source where `package.json` is found. -Depending on how you install the package, you may need to adhere to the naming convention. +```yaml +npmScopes: + <my-org>: + npmPublishRegistry: 'https://<your_domain>/api/v4/projects/<your_project_id>/packages/npm/' + npmAlwaysAuth: true + npmAuthToken: '<your_token>' +``` -You can use one of two API endpoints to install packages: +In this configuration: -- **Instance-level**: Use when you have many npm packages in different GitLab groups or in their own namespace. -- **Project-level**: Use when you have a few npm packages, and they are not in the same GitLab group. +- Replace `<my-org>` with your organization scope, exclude the `@` symbol. +- Replace `<your_domain>` with your domain name. +- Replace `<your_project_id>` with your project's ID, which you can find on the project's home page. +- Replace `<your_token>` with a deployment token, group access token, project access token, or personal access token. -If you plan to install a package through the [project level](#install-from-the-project-level), you do not have to -adhere to the naming convention. +Scoped registry does not work in Yarn Classic in `package.json` file, based on +this [issue](https://github.com/yarnpkg/yarn/pull/7829). +Therefore, under `publishConfig` there should be `registry` and not `@scope:registry` for Yarn Classic. +You can publish using your command line or a CI/CD pipeline to the GitLab Package Registry. -If you plan to install a package through the [instance level](#install-from-the-instance-level), then you must name -your package with a [scope](https://docs.npmjs.com/misc/scope/). Scoped packages begin with a `@` and have the -`@owner/package-name` format. You can set up the scope for your package in the `.yarnrc.yml` file and by using the -`publishConfig` option in the `package.json`. +### Publishing via the command line - Manual Publish -- The value used for the `@scope` is the root of the project that hosts the packages and not the root - of the project with the package's source code. The scope should be lowercase. -- The package name can be anything you want +```shell +# Yarn 1 (Classic) +yarn publish -| Project URL | Package Registry in | Scope | Full package name | -| ------------------------------------------------------- | ------------------- | --------- | ---------------------- | -| `https://gitlab.com/my-org/engineering-group/analytics` | Analytics | `@my-org` | `@my-org/package-name` | +# Yarn 2+ +yarn npm publish +``` -### Configuring `.yarnrc.yml` to publish from the project level +Your package should now publish to the Package Registry. -To publish with the project-level npm endpoint, set the following configuration in -`.yarnrc.yml`: +### Publishing via a CI/CD pipeline - Automated Publish -```yaml -npmScopes: - foo: - npmRegistryServer: 'https://<your_domain>/api/v4/projects/<your_project_id>/packages/npm/' - npmPublishRegistry: 'https://<your_domain>/api/v4/projects/<your_project_id>/packages/npm/' +You can use pipeline variables when you use this method. -npmRegistries: - //gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/: - npmAlwaysAuth: true - npmAuthToken: '<your_token>' -``` +You can use **Shared Runners** *(Default)* or **Private Runners** (Advanced). -In this configuration: +#### Shared runners -- Replace `<your_domain>` with your domain name. -- Replace `<your_project_id>` with your project's ID, which you can find on the project's home page. -- Replace `<your_token>` with a deploy token, group access token, project access token, or personal access token. +Third party images such as `node:latest` or `node:current` do not have direct access +to the `CI_JOB_TOKEN` when operating in a shared runner. You must configure an +authentication token or use a private runner. + +To create a authentication token: -### Configuring `.yarnrc.yml` to publish from the instance level +1. On the top bar, select **Main menu**, and: + - For a project, select **Projects** and find your project. + - For a group, select **Groups** and find your group. +1. On the left sidebar, select **Settings > Repository > Deploy Tokens**. +1. Create a deployment token with `read_package_registry` and `write_package_registry` scopes and copy the generated token. +1. On the left sidebar, select **Settings > CI/CD > Variables**. +1. Select `Add variable` and use the following settings: -For the instance-level npm endpoint, use this Yarn 2 configuration in `.yarnrc.yml`: +| Field | Value | +|--------------------|------------------------------| +| key | `NPM_AUTH_TOKEN` | +| value | `<DEPLOY-TOKEN-FROM-STEP-3>` | +| type | Variable | +| Protected variable | `CHECKED` | +| Mask variable | `CHECKED` | +| Expand variable | `CHECKED` | + +To use any **Protected variable**: + + 1. Go to the repository that contains the Yarn package source code. + 1. On the left sidebar, select **Settings > Repository**. + - If you are building from branches with tags, select **Protected Tags** and add `v*` (wildcard) for semantic versioning. + - If you are building from branches without tags, select **Protected Branches**. + +Then add the `NPM_AUTH_TOKEN` created above, to the `.yarnrc.yml` configuration +in your package project root directory where `package.json` is found: ```yaml npmScopes: - <scope>: - npmRegistryServer: 'https://<your_domain>/api/v4/packages/npm/' - -npmRegistries: - //gitlab.example.com/api/v4/packages/npm/: + esp-code: + npmPublishRegistry: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/" npmAlwaysAuth: true - npmAuthToken: '<your_token>' + npmAuthToken: "${NPM_AUTH_TOKEN}" ``` -In this configuration: +#### Private runners -- Replace `<your_domain>` with your domain name. -- Your scope is `<scope>`, without `@`. -- Replace `<your_token>` with a deploy token, group access token, project access token, or personal access token. +Add the `CI_JOB_TOKEN` to the `.yarnrc.yml` configuration in your package project +root directory where `package.json` is found: -### Publishing a package via the command line +```yaml +npmScopes: + esp-code: + npmPublishRegistry: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/" + npmAlwaysAuth: true + npmAuthToken: "${CI_JOB_TOKEN}" +``` -Publish a package: +To publish the package using CI/CD pipeline, In the GitLab project that houses +your `yarnrc.yml`, edit or create a `.gitlab-ci.yml` file. For example to trigger +only on any tag push: -```shell -npm publish -``` +```yaml +# Yarn 1 +image: node:lts -Your package should now publish to the Package Registry. +stages: + - deploy -### Publishing via a CI/CD pipeline +rules: +- if: $CI_COMMIT_TAG -In the GitLab project that houses your `yarnrc.yml`, edit or create a `.gitlab-ci.yml` file. For example: +deploy: + stage: deploy + script: + - yarn publish +``` ```yaml -image: node:latest +# Yarn 2+ +image: node:lts stages: - deploy +rules: + - if: $CI_COMMIT_TAG + deploy: stage: deploy + before_script: + - corepack enable + - yarn set version stable script: - - npm publish + - yarn npm publish ``` Your package should now publish to the Package Registry when the pipeline runs. ## Install a package -If multiple packages have the same name and version, the most recently-published package is retrieved when you install a package. +NOTE: +If multiple packages have the same name and version, the most recently-published +package is retrieved when you install a package. -You can install a package from a GitLab project or instance: +You can use one of two API endpoints to install packages: -- **Instance-level**: Use when you have many npm packages in different GitLab groups or in their own namespace. -- **Project-level**: Use when you have a few npm packages, and they are not in the same GitLab group. +- **Instance-level**: Best used when working with many packages in an organization scope. -### Install from the instance level +- If you plan to install a package through the [instance level](#install-from-the-instance-level), + then you must name your package with a [scope](https://docs.npmjs.com/misc/scope/). + Scoped packages begin with a `@` and have the `@owner/package-name` format. You can set up + the scope for your package in the `.yarnrc.yml` file and by using the `publishConfig` + option in the `package.json`. -WARNING: -You must use packages published with the scoped [naming convention](#naming-convention) when you install a package from the instance level. +- The value used for the `@scope` is the organization root (top-level project) `...com/my-org` + *(@my-org)* that hosts the packages, not the root of the project with the package's source code. +- The scope is always lowercase. +- The package name can be anything you want `@my-org/any-name`. -1. Authenticate to the Package Registry +- **Project-level**: For when you have a one-off package. - If you install a package from a private project, you must authenticate to the Package Registry. Skip this step if the project is not private. +If you plan to install a package through the [project level](#install-from-the-project-level), +you do not have to adhere to the naming convention. - ```shell - npm config set -- //your_domain_name/api/v4/packages/npm/:_authToken=your_token - ``` +| Project URL | Package Registry | Organization Scope | Full package name | +|-------------------------------------------------------------------|----------------------|--------------------|-----------------------------| +| `https://gitlab.com/<my-org>/<group-name>/<package-name-example>` | Package Name Example | `@my-org` | `@my-org/package-name` | +| `https://gitlab.com/<example-org>/<group-name>/<project-name>` | Project Name | `@example-org` | `@example-org/project-name` | - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +You can install from the instance level or from the project level. -1. Set the registry +The configurations for `.yarnrc.yml` can be added per package consuming project +root where `package.json` is located, or you can use a global +configuration located in your system user home directory. - ```shell - npm config set @scope:registry https://your_domain_name.com/api/v4/packages/npm/ - ``` +### Install from the instance level - - Replace `@scope` with the [root level group](#naming-convention) of the project you're installing to the package from. - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +Use these steps for global configuration in the `.yarnrc.yml` file: -1. Install the package +1. [Configure organization scope](#configure-organization-scope). +1. [Set the registry](#set-the-registry). - ```shell - yarn add @scope/my-package - ``` +#### Configure organization scope + +```yaml +npmScopes: + <my-org>: + npmRegistryServer: "https://<your_domain_name>/api/v4/packages/npm" +``` + +- Replace `<my-org>` with the root level group of the project you're installing to the package from excluding the `@` symbol. +- Replace `<your_domain_name>` with your domain name, for example, `gitlab.com`. + +#### Set the registry + +Skip this step if your package is public not private. + +```yaml + npmRegistries: + //<your_domain_name>/api/v4/packages/npm: + npmAlwaysAuth: true + npmAuthToken: "<your_token>" +``` + +- Replace `<your_domain_name>` with your domain name, for example, `gitlab.com`. +- Replace `<your_token>` with a deployment token (recommended), group access token, project access token, or personal access token. ### Install from the project level -1. Authenticate to the Package Registry +Use these steps for each project in the `.yarnrc.yml` file: + +1. [Configure project scope](#configure-project-scope). +1. [Set the registry](#set-the-registry-project-level). - If you install a package from a private project, you must authenticate to the Package Registry. Skip this step if the project is not private. +#### Configure project scope - ```shell - npm config set -- //your_domain_name/api/v4/projects/your_project_id/packages/npm/:_authToken=your_token - ``` + ```yaml + npmScopes: + <my-org>: + npmRegistryServer: "https://<your_domain_name>/api/v4/projects/<your_project_id>/packages/npm" +``` - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_project_id` is your project ID, found on the project's home page. - - Replace `your_token` with a deploy token, group access token, project access token, or personal access token. +- Replace `<my-org>` with the root level group of the project you're installing to the package from excluding the `@` symbol. +- Replace `<your_domain_name>` with your domain name, for example, `gitlab.com`. +- Replace `<your_project_id>` with your project ID, found on the project's home page. -1. Set the registry +#### Set the registry (project level) - ```shell - npm config set @scope:registry=https://your_domain_name/api/v4/projects/your_project_id/packages/npm/ - ``` +Skip this step if your package is public not private. - - Replace `@scope` with the [root level group](#naming-convention) of the project you're installing to the package from. - - Replace `your_domain_name` with your domain name, for example, `gitlab.com`. - - Replace `your_project_id` is your project ID, found on the project's home page. +```yaml +npmRegistries: + //<your_domain_name>/api/v4/projects/<your_project_id>/packages/npm: + npmAlwaysAuth: true + npmAuthToken: "<your_token>" +``` -1. Install the package +- Replace `<your_domain_name>` with your domain name, for example, `gitlab.com`. +- Replace `<your_token>` with a deployment token (recommended), group access token, project access token, or personal access token. +- Replace `<your_project_id>` with your project ID, found on the project's home page. - ```shell - yarn add @scope/my-package - ``` +### Install the package -## Helpful hints +For Yarn 2+, use `yarn add` either in the command line or in the CI/CD pipelines to install your packages: -For full helpful hints information, refer to the [npm documentation](../npm_registry/index.md#helpful-hints). +```shell +yarn add @scope/my-package +``` -### Supported CLI commands +#### For Yarn Classic -The GitLab npm repository supports the following commands for the npm CLI (`npm`) and yarn CLI -(`yarn`): +The Yarn Classic setup, requires both `.npmrc` and `.yarnrc` files as +[mentioned in issue](https://github.com/yarnpkg/yarn/issues/4451#issuecomment-753670295): -- `npm install`: Install npm packages. -- `npm publish`: Publish an npm package to the registry. -- `npm dist-tag add`: Add a dist-tag to an npm package. -- `npm dist-tag ls`: List dist-tags for a package. -- `npm dist-tag rm`: Delete a dist-tag. -- `npm ci`: Install npm packages directly from your `package-lock.json` file. -- `npm view`: Show package metadata. -- `yarn add`: Install an npm package. -- `yarn update`: Update your dependencies. +- Place credentials in the `.npmrc` file. +- Place the scoped registry in the `.yarnrc` file. -## Troubleshooting +```shell +# .npmrc +//<your_domain_name>/api/v4/projects/<your_project_id>/packages/npm/:_authToken="<your_token>" + +# .yarnrc +"@scope:registry" "https://<your_domain_name>/api/v4/projects/<your_project_id>/packages/npm/" +``` + +Then you can use `yarn add` to install your packages. + +## Related topics -For full troubleshooting information, refer to the [npm documentation](../npm_registry/index.md#troubleshooting). +- [npm documentation](../npm_registry/index.md#helpful-hints) +- [Yarn Migration Guide](https://yarnpkg.com/getting-started/migration) + +## Troubleshooting ### Error running Yarn with the Package Registry for the npm registry -If you are using [Yarn](https://classic.yarnpkg.com/en/) with the npm registry, you may get -an error message like: +If you are using [Yarn](https://classic.yarnpkg.com/en/) with the npm registry, you may get an error message like: ```shell yarn install v1.15.2 @@ -233,14 +323,7 @@ info If you think this is a bug, please open a bug report with the information p info Visit https://classic.yarnpkg.com/en/docs/cli/install for documentation about this command ``` -In this case, try adding this to your `.npmrc` file (and replace `<your_token>` -with your personal access token or deploy token): - -```plaintext -//gitlab.example.com/api/v4/projects/:_authToken=<your_token> -``` - -You can also use `yarn config` instead of `npm config` when setting your auth-token dynamically: +In this case, the following commands creates a file called `.yarnrc` in the current directory. Make sure to be in either your user home directory for global configuration or your project root for per-project configuration: ```shell yarn config set '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 8e736b6d83e..418c01cd851 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -23,6 +23,7 @@ The available roles are: - Developer - Maintainer - Owner +- Minimal Access (available for the top-level group only) A user assigned the Guest role has the least permissions, and the Owner has the most. @@ -58,7 +59,7 @@ The following table lists project permissions available for each role: |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----------|-----------|------------|----------| | [Analytics](analytics/index.md):<br>View [issue analytics](analytics/issue_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [value stream analytics](analytics/value_stream_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [value stream analytics](group/value_stream_analytics/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [DORA metrics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [CI/CD analytics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) | | ✓ | ✓ | ✓ | ✓ | @@ -69,8 +70,8 @@ The following table lists project permissions available for each role: | [Application security](application_security/index.md):<br>View [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>Create a [CVE ID Request](application_security/cve_id_request.md) | | | | ✓ | ✓ | | [Application security](application_security/index.md):<br>Create or assign [security policy project](application_security/policies/index.md) | | | | | ✓ | -| [Clusters](infrastructure/clusters/index.md):<br>View clusters | | | ✓ | ✓ | ✓ | -| [Clusters](infrastructure/clusters/index.md):<br>Manage clusters | | | | ✓ | ✓ | +| [GitLab Agent for Kubernetes](clusters/agent/index.md):<br>View agents | | | ✓ | ✓ | ✓ | +| [GitLab Agent for Kubernetes](clusters/agent/index.md):<br>Manage agents | | | | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete [cleanup policies](packages/container_registry/delete_container_registry_images.md#use-a-cleanup-policy) | | | | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Push an image to the Container Registry | | | ✓ | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (19) | ✓ (19) | ✓ | ✓ | ✓ | @@ -102,6 +103,8 @@ The following table lists project permissions available for each role: | [Issues](project/issues/index.md):<br>View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>View [related issues](project/issues/related_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Set [weight](project/issues/issue_weight.md) | ✓ (15) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Set metadata such as labels, milestones, or assignees when creating an issue | ✓ (15) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Edit metadata such labels, milestones, or assignees for an existing issue | (15) | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Set [parent epic](group/epics/manage_epics.md#add-an-existing-issue-to-an-epic) | | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>View [confidential issues](project/issues/confidential_issues.md) | (2) | ✓ | ✓ | ✓ | ✓ | | [Issues](project/issues/index.md):<br>Close / reopen (18) | | ✓ | ✓ | ✓ | ✓ | @@ -137,7 +140,7 @@ The following table lists project permissions available for each role: | [Package registry](packages/index.md):<br>Delete a package | | | | ✓ | ✓ | | [Package registry](packages/index.md):<br>Delete a file associated with a package | | | | ✓ | ✓ | | [Project operations](../operations/index.md):<br>View [Error Tracking](../operations/error_tracking.md) list | | ✓ | ✓ | ✓ | ✓ | -| [Project operations](../operations/index.md):<br>Manage [Feature Flags](../operations/feature_flags.md) | | | ✓ | ✓ | ✓ | +| [Project operations](../operations/index.md):<br>Manage [Feature flags](../operations/feature_flags.md) | | | ✓ | ✓ | ✓ | | [Project operations](../operations/index.md):<br>Manage [Error Tracking](../operations/error_tracking.md) | | | | ✓ | ✓ | | [Projects](project/index.md):<br>Download project | ✓ (1) | ✓ | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -220,7 +223,7 @@ The following table lists project permissions available for each role: <!-- markdownlint-disable MD029 --> -1. On self-managed GitLab instances, users with the Guest role are able to perform this action only on public and internal projects (not on private projects). [External users](admin_area/external_users.md) must be given explicit access even if the project is internal. Users with the Guest role on GitLab.com are only able to perform this action on public projects because internal visibility is not available. In GitLab 15.9 and later, this restriction only applies to users with the non-custom Guest role on self-managed GitLab instances and GitLab.com. +1. On self-managed GitLab instances, users with the Guest role are able to perform this action only on public and internal projects (not on private projects). [External users](admin_area/external_users.md) must be given explicit access even if the project is internal. Users with the Guest role on GitLab.com are only able to perform this action on public projects because internal visibility is not available. 2. Guest users can only view the [confidential issues](project/issues/confidential_issues.md) they created themselves or are assigned to. 3. Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [protected branches](project/protected_branches.md). 4. If the [branch is protected](project/protected_branches.md), this depends on the access given to Developers and Maintainers. @@ -271,8 +274,7 @@ More details about the permissions for some project-level features follow. | View and download artifacts | ✓ (1) | ✓ (2) | ✓ | ✓ | ✓ | ✓ | | View [environments](../ci/environments/index.md) | ✓ (3) | ✓ (3) | ✓ | ✓ | ✓ | ✓ | | View job logs and job details page | ✓ (1) | ✓ (2) | ✓ | ✓ | ✓ | ✓ | -| View pipeline details page | ✓ (1) | ✓ (2) | ✓ | ✓ | ✓ | ✓ | -| View pipelines page | ✓ (1) | ✓ (2) | ✓ | ✓ | ✓ | ✓ | +| View pipelines and pipeline details pages | ✓ (1) | ✓ (2) | ✓ | ✓ | ✓ | ✓ | | View pipelines tab in MR | ✓ (3) | ✓ (3) | ✓ | ✓ | ✓ | ✓ | | [View vulnerabilities in a pipeline](application_security/vulnerability_report/pipeline.md#view-vulnerabilities-in-a-pipeline) | | ✓ (2) | ✓ | ✓ | ✓ | ✓ | | View and download project-level [Secure Files](../api/secure_files.md) | | | | ✓ | ✓ | ✓ | @@ -328,7 +330,7 @@ This table shows granted privileges for jobs triggered by specific types of user | Push source and LFS | | | | | 1. Only if the triggering user is not an external one. -1. Only if the triggering user is a member of the project. See also [Usage of private Docker images with `if-not-present` pull policy](http://docs.gitlab.com/runner/security/index.html#usage-of-private-docker-images-with-if-not-present-pull-policy). +1. Only if the triggering user is a member of the project. See also [Usage of private Docker images with `if-not-present` pull policy](https://docs.gitlab.com/runner/security/index.html#usage-of-private-docker-images-with-if-not-present-pull-policy). ## Group members permissions @@ -424,39 +426,38 @@ nested groups if you have membership in one of its parents. For more information, see [subgroup memberships](group/subgroups/index.md#subgroup-membership). -## Users with minimal access **(PREMIUM)** +## Users with Minimal Access **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in GitLab 13.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in GitLab 13.4. +> - Support for inviting users with Minimal Access role [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106438) in GitLab 15.9. -Owners can add members with a "minimal access" role to a root group. Such users don't automatically have access to -projects and subgroups underneath. Owners must explicitly add these "minimal access" users to the specific subgroups and +Users with the Minimal Access role do not: + +- Count as licensed seats on self-managed Ultimate subscriptions or any GitLab.com subscriptions. +- Automatically have access to projects and subgroups in that root group. + +Owners must explicitly add these users to the specific subgroups and projects. -You can use minimal access to give the same member more than one role in a group: +You can use the Minimal Access role to give the same member more than one role in a group: -1. Add the member to the root group with a minimal access role. +1. Add the member to the root group with a Minimal Access role. 1. Invite the member as a direct member with a specific role in any subgroup or project in that group. -Because of an [outstanding issue](https://gitlab.com/gitlab-org/gitlab/-/issues/267996), when minimal access users: +Because of an [outstanding issue](https://gitlab.com/gitlab-org/gitlab/-/issues/267996), when a user with the Minimal Access role: -- Sign in with standard web authentication, they receive a `404` error when accessing the parent group. -- Sign in with Group SSO, they receive a `404` error immediately because they are redirected to the parent group page. +- Signs in with standard web authentication, they receive a `404` error when accessing the parent group. +- Signs in with Group SSO, they receive a `404` error immediately because they are redirected to the parent group page. To work around the issue, give these users the Guest role or higher to any project or subgroup within the parent group. -### Minimal access users take license seats - -Users with even a "minimal access" role are counted against your number of license seats. This -requirement does not apply for [GitLab Ultimate](https://about.gitlab.com/pricing/) -subscriptions. - ## Related topics - [The GitLab principles behind permissions](https://about.gitlab.com/handbook/product/gitlab-the-product/#permissions-in-gitlab) - [Members](project/members/index.md) - Customize permissions on [protected branches](project/protected_branches.md) - [LDAP user permissions](group/access_and_permissions.md#manage-group-memberships-via-ldap) -- [Value stream analytics permissions](analytics/value_stream_analytics.md#access-permissions-for-value-stream-analytics) +- [Value stream analytics permissions](group/value_stream_analytics/index.md#access-permissions-for-value-stream-analytics) - [Project aliases](../user/project/import/index.md#project-aliases) - [Auditor users](../administration/auditor_users.md) - [Confidential issues](project/issues/confidential_issues.md) @@ -467,9 +468,10 @@ subscriptions. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106256) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `customizable_roles`. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110810) in GitLab 15.9. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114524) in GitLab 15.10. Custom roles allow group members who are assigned the Owner role to create roles -specific to the needs of their organization. +specific to the needs of their organization. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a demo of the custom roles feature, see [[Demo] Ultimate Guest can view code on private repositories via custom role](https://www.youtube.com/watch?v=46cp_-Rtxps). @@ -481,29 +483,36 @@ To enable custom roles for your group, a group member with the Owner role: 1. Makes sure that there is at least one private project in this group or one of its subgroups, so that you can see the effect of giving a Guest a custom role. 1. Creates a personal access token with the API scope. -1. Uses [the API](../api/member_roles.md#add-a-member-role-to-a-group) to create the Guest+1 role for the group. +1. Uses [the API](../api/member_roles.md#add-a-member-role-to-a-group) to create the Guest+1 role for the root group. ### Associate a custom role with an existing group member To associate a custom role with an existing group member, a group member with the Owner role: -1. Invites a test user account to the root group as a Guest. - At this point, this Guest user cannot see any code on the projects in the group. +1. Invites a user to the root group or any subgroup or project in the root + group's hierarchy as a Guest. At this point, this Guest user cannot see any + code on the projects in the group or subgroup. 1. Optional. If the Owner does not know the `ID` of the Guest user receiving a custom role, finds that `ID` by making an [API request](../api/member_roles.md#list-all-member-roles-of-a-group). -1. Associates the group member with the Guest+1 role using the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) +1. Associates the member with the Guest+1 role using the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) + + ```shell + # to update a project membership + curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer $YOUR_ACCESS_TOKEN" --data '{"member_role_id": '$MEMBER_ROLE_ID', "access_level": 10}' "https://example.gitlab.com/api/v4/projects/$ID/members/$GUEST_USER_ID" - ```shell - curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer $YOUR_ACCESS_TOKEN" --data '{"member_role_id": '$MEMBER_ROLE_ID', "access_level": 10}' "https://example.gitlab.com/api/v4/groups/$GROUP_PATH/members/$GUEST_USER_ID" - ``` + # to update a group membership + curl --request PUT --header "Content-Type: application/json" --header "Authorization: Bearer $YOUR_ACCESS_TOKEN" --data '{"member_role_id": '$MEMBER_ROLE_ID', "access_level": 10}' "https://example.gitlab.com/api/v4/groups/$ID/members/$GUEST_USER_ID" + ``` Where: + + - `$ID`: The `ID` or [URL-encoded path of the project or group](../api/rest/index.md#namespaced-path-encoding) associated with the membership receiving the custom role. - `$MEMBER_ROLE_ID`: The `ID` of the member role created in the previous section. - `$GUEST_USER_ID`: The `ID` of the Guest user receiving a custom role. - Now the Guest+1 user can view code on all projects in the root group. + Now the Guest+1 user can view code on all projects associated with this membership. ### Remove a custom role from a group member @@ -532,8 +541,5 @@ the Owner role: ### Known issues - Additional permissions can only be applied to users with the Guest role. -- There is no visual distinction in the UI between the Guest role and the Guest role with additional permission. For more information, see [issue 384099](https://gitlab.com/gitlab-org/gitlab/-/issues/384099). - If a user with a custom role is shared with a group or project, their custom role is not transferred over with them. The user has the regular Guest role in the new group or project. -- If a custom role is deleted, the users associated with that custom role are also removed from the group. For more information, see [issue 370352](https://gitlab.com/gitlab-org/gitlab/-/issues/370352). -- The API endpoint for associating a custom role with a user only works for users with the Guest role in a group. A project member can be associated with a custom role, but not through the API yet. For more information, see [issue 385495](https://gitlab.com/gitlab-org/gitlab/-/issues/385495). -- The only way to remove a custom role from a user's membership to a Group is to delete the custom role, which deletes the user membership entirely. See [issue 387769](https://gitlab.com/gitlab-org/gitlab/-/issues/387769). +- You cannot use an [Auditor user](../administration/auditor_users.md) as a template for a custom role. diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md index 6d6a609618b..b5f936e7848 100644 --- a/doc/user/product_analytics/index.md +++ b/doc/user/product_analytics/index.md @@ -4,20 +4,30 @@ group: Product Analytics info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Product analytics **(ULTIMATE)** +# Product analytics (Experiment) **(ULTIMATE)** -> - Introduced in GitLab 15.4 as an [Alpha](../../policy/alpha-beta-support.md#alpha-features) feature [with a flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. +> - Introduced in GitLab 15.4 as an [Experiment](../../policy/alpha-beta-support.md#experiment) feature [with a flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. > - `cube_api_proxy` revised to only reference the [Product Analytics API](../../api/product_analytics.md) in GitLab 15.6. +> - `cube_api_proxy` removed and replaced with `product_analytics_internal_preview` in GitLab 15.10. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `cube_api_proxy`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. This page is a work in progress, and we're updating the information as we add more features. For more information, see the [group direction page](https://about.gitlab.com/direction/analytics/product-analytics/). -## How Product Analytics works +## How product analytics works + +Product analytics uses several tools: + +- [**Jitsu**](https://jitsu.com/docs) - A web and app event collection platform that provides a consistent API to collect user data and pass it through to ClickHouse. +- [**ClickHouse**](https://clickhouse.com/docs) - A database suited to store, query, and retrieve analytical data. +- [**Cube.js**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in Clickhouse. + +The following diagram illustrates the product analytics flow: ```mermaid --- @@ -26,7 +36,7 @@ title: Product Analytics flow flowchart TB subgraph Adding data A([SDK]) --Send user data--> B[Analytics Proxy] - B --Transform data and pass it through--> C[Jitsu] + B --Transform data and pass it through--> C[Snowplow] C --Pass the data to the associated database--> D([Clickhouse]) end subgraph Showing dashboards @@ -40,24 +50,20 @@ flowchart TB end ``` -Product Analytics uses several tools: - -- [**Jitsu**](https://jitsu.com/docs) - A web and app event collection platform that provides a consistent API to collect user data and pass it through to Clickhouse. -- [**Clickhouse**](https://clickhouse.com/docs) - A database suited to store, query, and retrieve analytical data. -- [**Cube.js**](https://cube.dev/docs/) - An analytical graphing library that provides an API to run queries against the data stored in Clickhouse. - ## Enable product analytics > - Introduced in GitLab 15.6 behind the [feature flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default. > - Moved to be behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_admin_settings` in GitLab 15.7. Disabled by default. +> - `cube_api_proxy` removed and replaced with `product_analytics_internal_preview` in GitLab 15.10. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_admin_settings`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flags](../../administration/feature_flags.md) named `product_analytics_dashboards` and `product_analytics_admin_settings`. On GitLab.com, this feature is not available. This feature is not ready for production use. -You can enable and configure product analytics to track events -within your project applications on a self-managed instance. +To track events in your project applications on a self-managed instance, +you must enable and configure product analytics. Prerequisite: @@ -65,35 +71,41 @@ Prerequisite: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > General**. -1. Expand the **Product analytics** section. +1. Expand the **Analytics** tab and find the **Product analytics** section. 1. Select **Enable product analytics** and enter the configuration values. The following table shows the required configuration parameters and example values: - | Name | Value | - |------------------------------|------------------------------------------------------------| - | Jitsu host | `https://jitsu.gitlab.com` | - | Jitsu project ID | `g0maofw84gx5sjxgse2k` | - | Jitsu administrator email | `jitsu.admin@gitlab.com` | - | Jitsu administrator password | `<your_password>` | - | Clickhouse URL | `https://<username>:<password>@clickhouse.gitlab.com:8123` | - | Cube API URL | `https://cube.gitlab.com` | - | Cube API key | `25718201b3e9...ae6bbdc62dbb` | + | Name | Value | + |--------------------------------|------------------------------------------------------------| + | Configurator connection string | `https://test:test@configurator.gitlab.com` | + | Jitsu host | `https://jitsu.gitlab.com` | + | Jitsu project ID | `g0maofw84gx5sjxgse2k` | + | Jitsu administrator email | `jitsu.admin@gitlab.com` | + | Jitsu administrator password | `<your_password>` | + | Collector host | `https://collector.gitlab.com` | + | ClickHouse URL | `https://<username>:<password>@clickhouse.gitlab.com:8123` | + | Cube API URL | `https://cube.gitlab.com` | + | Cube API key | `25718201b3e9...ae6bbdc62dbb` | 1. Select **Save changes**. ## Product analytics dashboards -> Introduced in GitLab 15.5 behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. Disabled by default. +> - Introduced in GitLab 15.5 behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. Disabled by default. +> - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_internal_preview`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. -Each project can define an unlimited number of dashboards. These dashboards are defined using our YAML schema and stored -in the `.gitlab/product_analytics/dashboards/` directory of a project repository. The name of the file is the name of the dashboard, and visualizations are shared across dashboards. +Each project can have an unlimited number of dashboards. +These dashboards are defined using the GitLab YAML schema, and stored in the `.gitlab/analytics/dashboards/` directory of a project repository. +The name of the file is the name of the dashboard. +Each dashboard can contain one or more visualizations (charts), which are shared across dashboards. -Project maintainers can enforce approval rules on dashboard changes using features such as code owners and approval rules. Dashboards are versioned in source control with the rest of a project's code. +Project maintainers can enforce approval rules on dashboard changes using features such as code owners and approval rules. +Dashboards are versioned in source control with the rest of a project's code. ### View project dashboards @@ -108,20 +120,28 @@ To view a list of product analytics dashboards for a project: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Dashboards**. +1. From the list of available dashboards, select the dashboard you want to view. ### Define a dashboard To define a dashboard: -1. In `.gitlab/product_analytics/dashboards/`, create a directory named like the dashboard. Each dashboard should have its own directory. -1. In the new directory, create a `.yaml` file with the same name as the directory. This file contains the dashboard definition, and must conform to the JSON schema defined in `ee/app/validators/json_schemas/product_analytics_dashboard.json`. -1. In the `.gitlab/product_analytics/dashboards/visualizations/` directory, create a `yaml` file. This file defines the visualization type for the dashboard, and must conform to the schema in +1. In `.gitlab/analytics/dashboards/`, create a directory named like the dashboard. + + Each dashboard should have its own directory. +1. In the new directory, create a `.yaml` file with the same name as the directory. + + This file contains the dashboard definition. It must conform to the JSON schema defined in `ee/app/validators/json_schemas/product_analytics_dashboard.json`. +1. In the `.gitlab/analytics/dashboards/visualizations/` directory, create a `.yaml` file. + + This file defines the visualization type for the dashboard. It must conform to the schema in `ee/app/validators/json_schemas/product_analytics_visualization.json`. -The example below includes three dashboards and one visualization that applies to all dashboards. +For example, if you want to create three dashboards (Conversion funnels, Demographic breakdown, and North star metrics) +and one visualization (line chart) that applies to all dashboards, the file structure would be: ```plaintext -.gitlab/product_analytics/dashboards +.gitlab/analytics/dashboards ├── conversion_funnels │ └── conversion_funnels.yaml ├── demographic_breakdown @@ -132,15 +152,40 @@ The example below includes three dashboards and one visualization that applies t │ └── example_line_chart.yaml ``` +### Define a chart visualization + +You can define different charts, and add visualization options to some of them: + +- Line chart, with the options listed in the [ECharts documentation](https://echarts.apache.org/en/option.html). +- Column chart, with the options listed in the [ECharts documentation](https://echarts.apache.org/en/option.html). +- Data table, with the only option to render `links` (array of objects, each with `text` and `href` properties to specify the dimensions to be used in links). See [example](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/validators/json_schemas/analytics_visualization.json?ref_type=heads#L112)). +- Single stat, with the only option to set `decimalPlaces` (number, default value is 0). + +To define a chart for your dashboards: + +1. In the `.gitlab/product_analytics/dashboards/visualizations/` directory, create a `.yaml` file. +The filename should be descriptive of the visualization it defines. +1. In the `.yaml` file, define the visualization options, according to the schema in +`ee/app/validators/json_schemas/analytics_visualization.json`. + +For [example](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/analytics/analytics_dashboards/gl_dashboards/product_analytics/visualizations/events_over_time.json), to create a line chart that illustrates event count over time, in the `visualizations` folder +create a `line_chart.yaml` file with the following required fields: + +- version +- title +- type +- data +- options + ## Funnel analysis -Funnel analysis can be used to understand the flow of users through your application and where +Use funnel analysis to understand the flow of users through your application, and where users drop out of a predefined flow (for example, a checkout process or ticket purchase). Each product can also define an unlimited number of funnels. -These funnels are defined using our YAML schema and stored in the `.gitlab/product_analytics/funnels/` directory of a project repository. +Like dashboards, funnels are defined using the GitLab YAML schema, and stored in the `.gitlab/analytics/funnels/` directory of a project repository. -Funnel definitions must include the keys `name`, `seconds_to_convert`, and an array of `steps`. +Funnel definitions must include the keys `name` and `seconds_to_convert`, and an array of `steps`. | Key | Description | |----------------------|----------------------------------------------------------| @@ -158,6 +203,8 @@ Each step must include the keys `name`, `target`, and `action`. ### Example funnel definition +The following example defines a funnel that tracks users who completed a purchase within one hour by going through three target pages: + ```yaml name: completed_purchase seconds_to_convert: 3600 @@ -205,3 +252,49 @@ The `afterDate` filter is not supported. Please use `beforeDate` or `inDateRange } } ``` + +## Raw data export + +Exporting the raw event data from the underlying storage engine can help you debug and create datasets for data analysis. + +Because Cube acts as an abstraction layer between the raw data and the API, the exported raw data has some caveats: + +- Data is grouped by the selected dimensions. Therefore, the exported data might be incomplete, unless including both `utcTime` and `userAnonymousId`. +- Data is by default limited to 10,000 rows, but you can increase the limit to maximum 50,000 rows. If your dataset has more than 50,000 rows, you must paginate through the results by using the `limit` and `offset` parameters. +- Data is always returned in JSON format. If you need it in a different format, you need to convert the JSON to the required format using a scripting language of your choice. + +[Issue 391683](https://gitlab.com/gitlab-org/gitlab/-/issues/391683) tracks efforts to implement a more scalable export solution. + +### Export raw data with Cube queries + +You can [query the raw data with the REST API](../../api/product_analytics.md#send-query-request-to-cube), +and convert the JSON output to any required format. + +To export the raw data for a specific dimension, pass a list of dimensions to the `dimensions` key. +For example, the following query outputs the raw data for the attributes listed: + +```json +POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi + +{ + "query":{ + "dimensions": [ + "TrackedEvents.docEncoding", + "TrackedEvents.docHost", + "TrackedEvents.docPath", + "TrackedEvents.docSearch", + "TrackedEvents.eventType", + "TrackedEvents.localTzOffset", + "TrackedEvents.pageTitle", + "TrackedEvents.src", + "TrackedEvents.utcTime", + "TrackedEvents.vpSize" + ], + "order": { + "TrackedEvents.apiKey": "asc" + } + } +} +``` + +If the request is successful, the returned JSON includes an array of rows of results. diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md index 60b8fe8ab88..f405dc42f46 100644 --- a/doc/user/profile/account/create_accounts.md +++ b/doc/user/profile/account/create_accounts.md @@ -16,9 +16,9 @@ You can create users: ## Create users on sign-in page -Prerequisites: +Prerequisite: -- [Sign-up enabled](../../admin_area/settings/sign_up_restrictions.md) +- [Sign-up must be enabled](../../admin_area/settings/sign_up_restrictions.md). Users can create their own accounts by either: @@ -27,18 +27,36 @@ Users can create their own accounts by either: ## Create users in Admin Area -Prerequisites: +Prerequisite: -- You must have administrator access for the instance. +- You must have administrator access to the instance. To create a user manually: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Users** (`/admin/users`). 1. Select **New user**. -1. Complete the fields. +1. Complete the required fields, such as name, username, and email. 1. Select **Create user**. +A reset link is sent to the user's email and they are forced to set their +password on first sign in. + +To set a user's password without relying on the email confirmation, after you +create a user following the previous steps: + +1. Select the user. +1. Select **Edit**. +1. Complete the password and password confirmation fields. +1. Select **Save changes**. + +The user can now sign in with the new username and password, and they are asked +to change the password you set up for them. + +NOTE: +If you wanted to create a test user, you could follow the previous steps +by providing a fake email and using the same password in the final confirmation. + ## Create users through authentication integrations Users are: diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index a74fd8d5215..a2c82fdeadf 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -48,9 +48,10 @@ When deleting users, you can either: - Delete just the user. Not all associated records are deleted with the user. Instead of being deleted, these records are moved to a system-wide user with the username Ghost User. The Ghost User's purpose is to act as a container for such records. Any commits made by a deleted user still display the username of the original user. + The user's personal projects are deleted, not moved to the Ghost User. - Delete the user and their contributions, including: - Abuse reports. - - Award emojis. + - Emoji reactions. - Epics. - Groups of which the user is the only user with the Owner role. - Issues. diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index b76b99b5242..8827e1e27c5 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -14,7 +14,7 @@ GitLab supports as a second factor of authentication: - Time-based one-time passwords ([TOTP](https://datatracker.ietf.org/doc/html/rfc6238)). When enabled, GitLab prompts you for a code when you sign in. Codes are generated by your one-time password authenticator (for example, a password manager on one of your devices). -- U2F or WebAuthn devices. You're prompted to activate your U2F or WebAuthn device (usually by pressing a button on it) when +- WebAuthn devices. You're prompted to activate your WebAuthn device (usually by pressing a button on it) when you supply your username and password to sign in. This performs secure authentication on your behalf. If you set up a device, also set up a TOTP so you can still access your account if you lose the device. @@ -24,28 +24,37 @@ If you set up a device, also set up a TOTP so you can still access your account When 2FA is enabled, you can't use your password to authenticate with Git over HTTPS or the [GitLab API](../../../api/rest/index.md). You can use a [personal access token](../personal_access_tokens.md) instead. -## Git Credential Manager +## OAuth credential helpers -For Git over HTTPS, [Git Credential Manager](https://github.com/GitCredentialManager/git-credential-manager) (GCM) offers an alternative to personal access tokens. By default, GCM -authenticates using OAuth, opening GitLab in your web browser. The first time you authenticate, GitLab asks you to authorize the app. If you remain signed in to GitLab, subsequent -authentication requires no interaction. +The following Git credential helpers authenticate to GitLab using OAuth. This is compatible with two-factor authentication. The first time you authenticate, the helper opens the web browser and GitLab asks you to authorize the app. Subsequent authentication requires no interaction. -So you don't need to reauthenticate on every push, GCM supports caching as well as a variety of platform-specific credential stores that persist between sessions. This feature is useful whether you use personal access tokens or OAuth. +### Git Credential Manager -GCM supports GitLab.com out the box. To use with self-managed GitLab, see [GitLab support](https://github.com/GitCredentialManager/git-credential-manager/blob/main/docs/gitlab.md) -documentation. +[Git Credential Manager](https://github.com/GitCredentialManager/git-credential-manager) (GCM) authenticates by default using OAuth. GCM supports GitLab.com without any manual configuration. To use GCM with self-managed GitLab, see [GitLab support](https://github.com/GitCredentialManager/git-credential-manager/blob/main/docs/gitlab.md). + +So you do not need to re-authenticate on every push, GCM supports caching as well as a variety of platform-specific credential stores that persist between sessions. This feature is useful whether you use personal access tokens or OAuth. + +Git for Windows includes Git Credential Manager. Git Credential Manager is developed primarily by GitHub, Inc. It is an open-source project and is supported by the community. +### git-credential-oauth + +[git-credential-oauth](https://github.com/hickford/git-credential-oauth) supports GitLab.com and several popular public hosts without any manual configuration needed. To use with self-managed GitLab, see the [git-credential-oauth custom hosts documentation](https://github.com/hickford/git-credential-oauth#custom-hosts). + +Many Linux distributions include git-credential-oauth as a package. + +git-credential-oauth is an open-source project supported by the community. + ## Enable two-factor authentication > - Account email confirmation requirement [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35102) in GitLab 14.3. [Deployed behind the `ensure_verified_primary_email_for_2fa` flag](../../../administration/feature_flags.md), enabled by default. > - Account email confirmation requirement generally available and [feature flag `ensure_verified_primary_email_for_2fa` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/340151) in GitLab 14.4. -You can enable 2FA: +You can enable 2FA using a: -- Using a one-time password authenticator. After you enable 2FA, back up your [recovery codes](#recovery-codes). -- Using a U2F or WebAuthn device. +- One-time password authenticator. After you enable 2FA, back up your [recovery codes](#recovery-codes). +- WebAuthn device. In GitLab 14.3 and later, your account email must be confirmed to enable 2FA. @@ -60,10 +69,11 @@ To enable 2FA with a one-time password: 1. **On your device (usually your phone):** 1. Install a compatible application. For example: - Cloud-based (recommended because you can restore access if you lose the hardware device): - - [Authy](https://authy.com/) + - [Authy](https://authy.com/). + - [Duo](https://duo.com/). - Other: - - [Google Authenticator](https://support.google.com/accounts/answer/1066447?hl=en) - - [Microsoft Authenticator](https://www.microsoft.com/en-us/security/mobile-authenticator-app) + - [Google Authenticator](https://support.google.com/accounts/answer/1066447?hl=en). + - [Microsoft Authenticator](https://www.microsoft.com/en-us/security/mobile-authenticator-app). 1. In the application, add a new entry in one of two ways: - Scan the code displayed by GitLab with your device's camera to add the entry automatically. - Enter the details provided to add the entry manually. @@ -72,9 +82,6 @@ To enable 2FA with a one-time password: 1. Enter your current password. 1. Select **Submit**. -NOTE: -DUO [cannot be used for 2FA](https://gitlab.com/gitlab-org/gitlab/-/issues/15760). - If you entered the correct pin, GitLab displays a list of [recovery codes](#recovery-codes). Download them and keep them in a safe place. @@ -141,6 +148,70 @@ Configure FortiAuthenticator in GitLab. On your GitLab server: 1. [Reconfigure](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) (Omnibus GitLab) or [restart](../../../administration/restart_gitlab.md#installations-from-source) (GitLab installed from source). +### Enable one-time password using Duo + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15760) in GitLab 15.10. + +FLAG: +On self-managed GitLab, by default this feature is available. On GitLab.com this feature is not available. + +You can use Duo as an OTP provider in GitLab. + +#### Prerequisites + +To use Duo as an OTP provider: + +- Your account must exist in both Duo and GitLab, with the same username in both applications. +- You must have [configured Duo](https://admin.duosecurity.com/) and have an integration key, secret key, and API hostname. + +For more information, see the [Duo API documentation](https://duo.com/docs/authapi). + +GitLab 15.10 has been tested with Duo version D261.14 + +#### Configure Duo in GitLab + +On your GitLab server: + +1. Open the configuration file. + + For Omnibus GitLab: + + ```shell + sudo editor /etc/gitlab/gitlab.rb + ``` + + For installations from source: + + ```shell + cd /home/git/gitlab + sudo -u git -H editor config/gitlab.yml + ``` + +1. Add the provider configuration: + + For Omnibus package: + + ```ruby + gitlab_rails['duo_auth_enabled'] = false + gitlab_rails['duo_auth_integration_key'] = '<duo_integration_key_value>' + gitlab_rails['duo_auth_secret_key'] = '<duo_secret_key_value>' + gitlab_rails['duo_auth_hostname'] = '<duo_api_hostname>' + ``` + + For installations from source: + + ```yaml + duo_auth: + enabled: true + hostname: <duo_api_hostname> + integration_key: <duo_integration_key_value> + secret_key: <duo_secret_key_value> + ``` + +1. Save the configuration file. +1. For Omnibus GitLab, [reconfigure GitLab](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + For installations from source, [restart GitLab](../../../administration/restart_gitlab.md#installations-from-source). + ### Enable one-time password using FortiToken Cloud > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212313) in GitLab 13.7 [with a flag](../../../administration/feature_flags.md) named `forti_token_cloud`. Disabled by default. @@ -198,68 +269,61 @@ Configure FortiToken Cloud in GitLab. On your GitLab server: 1. [Reconfigure](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) (Omnibus GitLab) or [restart](../../../administration/restart_gitlab.md#installations-from-source) (GitLab installed from source). -### Set up a U2F device - -GitLab officially supports [YubiKey](https://www.yubico.com/products/) U2F devices, but users have successfully used -[SoloKeys](https://solokeys.com/) and [Google Titan Security Key](https://cloud.google.com/titan-security-key). - -U2F is [supported by](https://caniuse.com/#search=U2F) the following desktop browsers: - -- Chrome -- Edge -- Opera -- Firefox 67+. For Firefox 47-66: - - 1. Enable the FIDO U2F API in [`about:config`](https://support.mozilla.org/en-US/kb/about-config-editor-firefox). - 1. Search for `security.webauth.u2f` and select it to toggle to `true`. - -To set up 2FA with a U2F device: - -1. Access your [**User settings**](../index.md#access-your-user-settings). -1. Select **Account**. -1. Select **Enable Two-Factor Authentication**. -1. Connect your U2F device. -1. Select **Set up New U2F Device**. -1. A light begins blinking on your device. Activate it by pressing its button. - -A message displays indicating that your device was successfully set up. Select **Register U2F Device** to complete the -process. Recovery codes are not generated for U2F devices. - ### Set up a WebAuthn device -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22506) in GitLab 13.4 [with a flag](../../../administration/feature_flags.md) named `webauthn`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/232671) in GitLab 14.6. +> - WebAuthn devices [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22506) in GitLab 13.4 [with a flag](../../../administration/feature_flags.md) named `webauthn`. Disabled by default. +> - WebAuthn devices [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/232671) in GitLab 14.6. +> - Optional one-time password authentication for WebAuthn devices [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378844) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `webauthn_without_topt`. [Enabled on GitLab.com and self-managed by default](https://gitlab.com/gitlab-org/gitlab/-/issues/232671). FLAG: -On self-managed GitLab, by default this feature is available. To disable the feature, ask an administrator to +On self-managed GitLab, by default, WebAuthn devices are available. To disable the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `webauthn`. If you disable the WebAuthn feature flag after WebAuthn devices have been registered, these devices are not usable until you re-enable this feature. +On GitLab.com, WebAuthn devices are available. + +FLAG: +On self-managed GitLab, by default, optional one-time password authentication for WebAuthn devices is not available. To enable the feature, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `webauthn_without_totp`. On GitLab.com, this feature is available. -WebAuthn [supported by](https://caniuse.com/#search=webauthn): +WebAuthn is [supported by](https://caniuse.com/#search=webauthn) the following: -- The following desktop browsers: +- Desktop browsers: - Chrome - Edge - Firefox - Opera - Safari -- The following mobile browsers: +- Mobile browsers: - Chrome for Android - Firefox for Android - iOS Safari (since iOS 13.3) To set up 2FA with a WebAuthn-compatible device: +1. Optional. [Set up a one-time password](#enable-one-time-password). 1. Access your [**User settings**](../index.md#access-your-user-settings). 1. Select **Account**. 1. Select **Enable Two-Factor Authentication**. 1. Plug in your WebAuthn device. +1. Enter a device name and in GitLab 15.10 and later, your GitLab account password. + You might not need to enter this password if you have signed in through your + identity provider. 1. Select **Set up New WebAuthn Device**. 1. Depending on your device, you might have to press a button or touch a sensor. -A message displays indicating that your device was successfully set up. Recovery codes are not generated for WebAuthn -devices. +You should receive a message indicating that you successfully set up your device. + +When you set up 2FA with a WebAuthn-compatible device, that device is linked to +a specific browser on a specific computer. Depending on the browser and WebAuthn +device, you might be able to configure settings to use the WebAuthn device on a +different browser or computer. + +If this is the first time you have set up 2FA, you +must [download recovery codes](#recovery-codes) so you can recover access to your +account if you lose access. + +WARNING: +You can lose access to your account if you clear your browser data. ## Recovery codes @@ -272,11 +336,11 @@ these recovery codes to sign in to your account. WARNING: Each code can be used only once to sign in to your account. -We recommend copying and printing them, or downloading them using the **Download codes** button for storage in a safe +You should copy and print the codes, or use **Download codes** to download them for storage in a safe place. If you choose to download them, the file is called `gitlab-recovery-codes.txt`. NOTE: -Recovery codes are not generated for U2F or WebAuthn devices. +Recovery codes are not generated for WebAuthn devices. If you lose the recovery codes, or want to generate new ones, you can use either: @@ -302,18 +366,7 @@ and you're presented with a second prompt, depending on which type of 2FA you've ### Sign in using a one-time password -When asked, enter the pin from your one time password authenticator's application or a recovery code to sign in. - -### Sign in using a U2F device - -To sign in by using a U2F device: - -1. Select **Login via U2F Device**. -1. A light begins blinking on your device. Activate it by touching/pressing - its button. - -A message displays indicating that your device responded to the authentication request, and you're automatically signed -in. +When asked, enter the pin from your one-time password authenticator's application or a recovery code to sign in. ### Sign in using a WebAuthn device @@ -333,7 +386,7 @@ To disable 2FA: 1. Under **Register Two-Factor Authenticator**, enter your current password and select **Disable two-factor authentication**. -This clears all your 2FA registrations, including mobile applications and U2F or WebAuthn devices. +This clears all your 2FA registrations, including mobile applications and WebAuthn devices. ## Recovery options @@ -412,18 +465,18 @@ a GitLab global administrator disable 2FA for your account: ## Information for GitLab administrators **(FREE SELF)** - Take care that 2FA keeps working after [restoring a GitLab backup](../../../raketasks/backup_restore.md). -- To ensure 2FA authorizes correctly with a time-based one time passwords (TOTP) server, synchronize your GitLab +- To ensure 2FA authorizes correctly with a time-based one-time password (TOTP) server, synchronize your GitLab server's time using a service like NTP. Otherwise, authorization can always fail because of time differences. -- The GitLab U2F and WebAuthn implementation does _not_ work when the GitLab instance is accessed from multiple hostnames - or FQDNs. Each U2F or WebAuthn registration is linked to the _current hostname_ at the time of registration, and +- The GitLab WebAuthn implementation does _not_ work when the GitLab instance is accessed from multiple hostnames + or FQDNs. Each WebAuthn registration is linked to the _current hostname_ at the time of registration, and cannot be used for other hostnames or FQDNs. For example, if a user is trying to access a GitLab instance from `first.host.xyz` and `second.host.xyz`: - - The user signs in by using `first.host.xyz` and registers their U2F key. - - The user signs out and attempts to sign in by using `first.host.xyz` - U2F authentication succeeds. - - The user signs out and attempts to sign in by using `second.host.xyz` - U2F authentication fails, because - the U2F key has only been registered on `first.host.xyz`. + - The user signs in by using `first.host.xyz` and registers their WebAuthn key. + - The user signs out and attempts to sign in by using `first.host.xyz` - WebAuthn authentication succeeds. + - The user signs out and attempts to sign in by using `second.host.xyz` - WebAuthn authentication fails, because + the WebAuthn key has only been registered on `first.host.xyz`. - To enforce 2FA at the system or group levels see, [Enforce two-factor authentication](../../../security/two_factor_authentication.md). @@ -441,23 +494,25 @@ access token instead of a password. This error occurs in the following scenarios: - You have 2FA enabled and have attempted to authenticate with a username and - password. For 2FA-enabled users, a [personal access token](../personal_access_tokens.md) (PAT) - must be used instead of a password. To authenticate: - - Git requests over HTTP(S), a PAT with `read_repository` or `write_repository` scope is required. - - [GitLab Container Registry](../../packages/container_registry/authenticate_with_container_registry.md) requests, a PAT - with `read_registry` or `write_registry` scope is required. - - [Dependency Proxy](../../packages/dependency_proxy/index.md#authenticate-with-the-dependency-proxy) requests, a PAT with - `read_registry` and `write_registry` scopes is required. + password. - You do not have 2FA enabled and have sent an incorrect username or password with your request. - You do not have 2FA enabled but an administrator has enabled the [enforce 2FA for all users](../../../security/two_factor_authentication.md#enforce-2fa-for-all-users) setting. - You do not have 2FA enabled, but an administrator has disabled the [password authentication enabled for Git over HTTP(S)](../../admin_area/settings/sign_in_restrictions.md#password-authentication-enabled) - setting. You can authenticate Git requests: - - Over HTTP(S) using a [personal access token](../personal_access_tokens.md). - - In your browser using [Git Credential Manager](#git-credential-manager). - - If you have configured LDAP, over HTTP(S) using an [LDAP password](../../../administration/auth/ldap/index.md). + setting. + +Instead you can authenticate: + +- Using a [personal access token](../personal_access_tokens.md) (PAT): + - For Git requests over HTTP(S), a PAT with `read_repository` or `write_repository` scope is required. + - For [GitLab Container Registry](../../packages/container_registry/authenticate_with_container_registry.md) requests, a PAT + with `read_registry` or `write_registry` scope is required. + - For [Dependency Proxy](../../packages/dependency_proxy/index.md#authenticate-with-the-dependency-proxy) requests, a PAT with + `read_registry` and `write_registry` scopes is required. +- If you have configured LDAP, using an [LDAP password](../../../administration/auth/ldap/index.md) +- Using an [OAuth credential helper](#oauth-credential-helpers). ### Error: "invalid pin code" diff --git a/doc/user/profile/achievements.md b/doc/user/profile/achievements.md new file mode 100644 index 00000000000..1313c714dd0 --- /dev/null +++ b/doc/user/profile/achievements.md @@ -0,0 +1,236 @@ +--- +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Achievements (Experiment) **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113156) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `achievements`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `achievements`. +The feature is not ready for production use. + +Achievements are a way to reward users for their activity on GitLab. +As a namespace maintainer or owner, you can create custom achievements for specific contributions, which you can award to or revoke from users based on your criteria. + +As a user, you can collect achievements to highlight your contributions to different projects or groups on your profile. +An achievement consists of a name, a description, and an avatar. + +![Achievements on user profile page](img/user_profile_achievements_v15_11.png) + +Achievements are considered to be owned by the user. They are visible regardless of the visibility setting of the namespace that created the Achievement. + +This feature is an Experiment. +For more information about planned work, see [epic 9429](https://gitlab.com/groups/gitlab-org/-/epics/9429). +Tell us about your use cases by leaving comments in the epic. + +## Types of achievement + +Programmatically, there is only one way to create, award, revoke, or delete achievements. + +Practically, you can differentiate between achievements that are awarded: + +- Once and irrevocable. For example, a "First contribution merged" achievement. +- Once and revocable. For example, a "Core team member" achievement. +- Multiple times. For example, a "Contributor of the month" achievement. + +## View a user's achievements + +You can view a user's achievements on their profile page. + +Prerequisites: + +- The user profile must be public. + +To view a user's achievements: + +1. Go to the user's profile page. +1. Below the user's avatar, see their achievements. +1. To view details about an achievement, hover over it. + It displays the following information: + + - Name of the achievement + - Description of the achievement + - Date when the achievement was awarded to the user + - Namespace that awarded the achievement if the user is a member of the namespace or the namespace is public + +To retrieve a list of a user's achievements, query the [`user` GraphQL type](../../api/graphql/reference/index.md#user). + +```graphql +query { + user(username: "<username>") { + userAchievements { + nodes { + achievement { + name + description + avatarUrl + namespace { + fullPath + name + } + } + } + } + } +} +``` + +## Create an achievement + +You can create custom achievements to award for specific contributions. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To create an achievement, call the [`achievementsCreate` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementscreate). + +```graphql +mutation achievementsCreate($file: Upload!) { + achievementsCreate( + input: { + namespaceId: "gid://gitlab/Namespace/<namespace id>", + name: "<name>", + description: "<description>", + avatar: $file} + ) { + errors + achievement { + id + name + description + avatarUrl + } + } +} +``` + +## Update an achievement + +You can change the name, description, and avatar of an achievement at any time. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To update an achievement, call the [`achievementsUpdate` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsupdate). + +```graphql +mutation achievementsUpdate($file: Upload!) { + achievementsUpdate( + input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>", + name: "<new name>", + description: "<new description>", + avatar: $file} + ) { + errors + achievement { + id + name + description + avatarUrl + } + } +} +``` + +## Award an achievement + +You can award an achievement to a user to recognize their contributions. +The user receives an email notification when they are awarded an achievement. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To award an achievement to a user, call the [`achievementsAward` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsaward). + +```graphql +mutation { + achievementsAward(input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>", + userId: "gid://gitlab/User/<user id>" }) { + userAchievement { + id + achievement { + id + name + } + user { + id + username + } + } + errors + } +} +``` + +## Revoke an achievement + +You can revoke a user's achievement if you consider the user no longer meets the awarding criteria. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To revoke an achievement, call the [`achievementsRevoke` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsrevoke). + +```graphql +mutation { + achievementsRevoke(input: { + userAchievementId: "gid://gitlab/Achievements::UserAchievement/<user achievement id>" }) { + userAchievement { + id + achievement { + id + name + } + user { + id + username + } + revokedAt + } + errors + } +} +``` + +## Delete an achievement + +If you consider you no longer need an achievement, you can delete it. +This will delete all related awarded and revoked instances of the achievement. + +Prerequisites: + +- You must have the Maintainer or Owner role for the namespace. + +To delete an achievement, call the [`achievementsDelete` GraphQL mutation](../../api/graphql/reference/index.md#mutationachievementsdelete). + +```graphql +mutation { + achievementsDelete(input: { + achievementId: "gid://gitlab/Achievements::Achievement/<achievement id>" }) { + achievement { + id + name + } + errors + } +} +``` + +## Hide achievements + +If you don't want to display achievements on your profile, you can opt out. To do this: + +1. On the top bar, in the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. In the **Main settings** section, clear the **Display achievements on your profile** checkbox. +1. Select **Update profile settings**. diff --git a/doc/user/profile/active_sessions.md b/doc/user/profile/active_sessions.md index 1f7264d740e..f22accf2ff5 100644 --- a/doc/user/profile/active_sessions.md +++ b/doc/user/profile/active_sessions.md @@ -40,8 +40,8 @@ To revoke an active session: NOTE: When any session is revoked all **Remember me** tokens for all -devices are revoked. See [Why do you keep getting signed out?](index.md#why-do-you-keep-getting-signed-out) -for more information about the **Remember me** feature. +devices are revoked. For details about **Remember me**, see +[cookies used for sign-in](index.md#cookies-used-for-sign-in). <!-- ## Troubleshooting diff --git a/doc/user/profile/comment_templates.md b/doc/user/profile/comment_templates.md new file mode 100644 index 00000000000..8a94aff44fe --- /dev/null +++ b/doc/user/profile/comment_templates.md @@ -0,0 +1,69 @@ +--- +stage: Create +group: Code Review +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Comment templates **(FREE)** + +> - GraphQL support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352956) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. +> - User interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113232) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `saved_replies`. Disabled by default. Enabled for GitLab team members only. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119468) in GitLab 16.0. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `saved_replies`. +On GitLab.com, this feature is not available by default, but enabled for GitLab team members. + +With comment templates, create and reuse text for any text area in: + +- Merge requests, including diffs. +- Issues, including design management comments. +- Epics. +- Work items. + +Comment templates can be small, like approving a merge request and unassigning yourself from it, +or large, like chunks of boilerplate text you use frequently: + +![Comment templates dropdown list](img/saved_replies_dropdown_v15_10.png) + +For more information about the rollout plan for this feature, see [issue 352956](https://gitlab.com/gitlab-org/gitlab/-/issues/352956). + +## Use comment templates in a text area + +To include the text of a comment template in your comment: + +1. In the editor toolbar for your comment, select **Comment templates** (**{symlink}**). +1. Select your desired comment template. + +## Create comment templates + +To create a comment template for future use: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Provide a **Name** for your comment template. +1. Enter the **Content** of your reply. You can use any formatting you use in + other GitLab text areas. +1. Select **Save**, and the page reloads with your comment template shown. + +## View your comment templates + +To go to your comment templates: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Scroll to **My comment templates**. + +## Edit or delete comment templates + +To edit or delete a previously comment template: + +1. On the top bar, in the upper-right corner, select your avatar. +1. From the dropdown list, select **Preferences**. +1. On the left sidebar, select **Comment templates** (**{symlink}**). +1. Scroll to **My comment templates**, and identify the comment template you want to edit. +1. To edit, select **Edit** (**{pencil}**). +1. To delete, select **Delete** (**{remove}**), then select **Delete** again from the modal window. diff --git a/doc/user/profile/contributions_calendar.md b/doc/user/profile/contributions_calendar.md index e9802cccef6..7b5691e1ee5 100644 --- a/doc/user/profile/contributions_calendar.md +++ b/doc/user/profile/contributions_calendar.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- @@ -83,7 +83,7 @@ GitLab provides RSS feeds of user activity. To subscribe to the RSS feed of a user's activity: 1. Go to the [user's profile](index.md#access-your-user-profile). -1. In the upper right, select the feed symbol **{rss}** to display the results as an RSS feed in Atom format. +1. In the upper-right corner, select the feed symbol (**{rss}**) to display the results as an RSS feed in Atom format. The URL of the result contains both a feed token, and the user's activity that you're authorized to view. diff --git a/doc/user/profile/img/busy_indicator_note_header_v13_9.png b/doc/user/profile/img/busy_indicator_note_header_v13_9.png Binary files differdeleted file mode 100644 index 63301ebdc14..00000000000 --- a/doc/user/profile/img/busy_indicator_note_header_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_notes_v13_9.png b/doc/user/profile/img/busy_indicator_notes_v13_9.png Binary files differdeleted file mode 100644 index 2efe075c72b..00000000000 --- a/doc/user/profile/img/busy_indicator_notes_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_profile_page_v13_6.png b/doc/user/profile/img/busy_indicator_profile_page_v13_6.png Binary files differdeleted file mode 100644 index c8e969f38db..00000000000 --- a/doc/user/profile/img/busy_indicator_profile_page_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png b/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png Binary files differdeleted file mode 100644 index 711638541dd..00000000000 --- a/doc/user/profile/img/busy_indicator_settings_menu_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png b/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png Binary files differdeleted file mode 100644 index 3dca88ec8cc..00000000000 --- a/doc/user/profile/img/busy_indicator_sidebar_collapsed_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_sidebar_v13_9.png b/doc/user/profile/img/busy_indicator_sidebar_v13_9.png Binary files differdeleted file mode 100644 index 83024b319bf..00000000000 --- a/doc/user/profile/img/busy_indicator_sidebar_v13_9.png +++ /dev/null diff --git a/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png b/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png Binary files differdeleted file mode 100644 index 16fb0e6556b..00000000000 --- a/doc/user/profile/img/busy_indicator_user_popovers_v13_6.png +++ /dev/null diff --git a/doc/user/profile/img/profile-preferences-syntax-themes.png b/doc/user/profile/img/profile-preferences-syntax-themes.png Binary files differdeleted file mode 100644 index 719df9410fc..00000000000 --- a/doc/user/profile/img/profile-preferences-syntax-themes.png +++ /dev/null diff --git a/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png b/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png Binary files differnew file mode 100644 index 00000000000..39cd35f3b5b --- /dev/null +++ b/doc/user/profile/img/profile-preferences-syntax-themes_v15_11.png diff --git a/doc/user/profile/img/saved_replies_dropdown_v15_10.png b/doc/user/profile/img/saved_replies_dropdown_v15_10.png Binary files differnew file mode 100644 index 00000000000..50313f71f4a --- /dev/null +++ b/doc/user/profile/img/saved_replies_dropdown_v15_10.png diff --git a/doc/user/profile/img/user_profile_achievements_v15_11.png b/doc/user/profile/img/user_profile_achievements_v15_11.png Binary files differnew file mode 100644 index 00000000000..cf675cd6b06 --- /dev/null +++ b/doc/user/profile/img/user_profile_achievements_v15_11.png diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index a84d16e6d0c..b7d8f6aba58 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -34,9 +34,13 @@ If you do not want to update the namespace, you can create a new user or group a Prerequisites: -- Your namespace cannot contain a project with [Container Registry](../packages/container_registry/index.md) tags. -- Your namespace cannot have a project that hosts [GitLab Pages](../project/pages/index.md). For more information, - see [this procedure in the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#change-your-username-at-gitlabcom). +- Your namespace must not: + - Contain a project with [Container Registry](../packages/container_registry/index.md) tags. + - Have a project that hosts [GitLab Pages](../project/pages/index.md). For more information, + see [changing your username in the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#change-your-username-at-gitlabcom). +- Your username must be between 2 and 255 characters in length, and must not: + - Contain special characters or emojis. + - End with `.<reserved file extension>`, for example `jon.png`. However, `jonpng` is valid. To change your username: @@ -131,11 +135,13 @@ To add links to other accounts: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Main settings** section, add your information from: - - Discord ([User ID](https://support.discord.com/hc/en-us/articles/206346498-Where-can-I-find-my-User-Server-Message-ID-)) - - LinkedIn - - Skype - - Twitter +1. In the **Main settings** section, add your: + - Discord [user ID](https://support.discord.com/hc/en-us/articles/206346498-Where-can-I-find-my-User-Server-Message-ID-). + - LinkedIn profile name. + - Skype username. + - Twitter @username. + + Your user ID or username must be 500 characters or less. 1. Select **Update profile settings**. ## Show private contributions on your user profile page @@ -160,7 +166,7 @@ To specify your pronouns: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Pronouns** text box, enter your pronouns. +1. In the **Pronouns** text box, enter your pronouns. The text must be 50 characters or less. 1. Select **Update profile settings**. ## Add your name pronunciation @@ -174,7 +180,7 @@ To add your name pronunciation: 1. On the top bar, in the upper-right corner, select your avatar. 1. Select **Edit profile**. -1. In the **Pronunciation** text box, enter how your name is pronounced. +1. In the **Pronunciation** text box, enter how your name is pronounced. The pronunciation must be plain text and 255 characters or less. 1. Select **Update profile settings**. ## Set your current status @@ -220,25 +226,7 @@ To set the busy status indicator, either: 1. Select **Edit profile**. 1. In the **Current status** section, select the **Set yourself as busy** checkbox. - The busy status is displayed in the user interface. - - Username: - - | Profile page | Settings menu | User popovers | - | --- | --- | --- | - | ![Busy status - profile page](img/busy_indicator_profile_page_v13_6.png) | ![Busy status - settings menu](img/busy_indicator_settings_menu_v13_6.png) | ![Busy status - user popovers](img/busy_indicator_user_popovers_v13_6.png) | - - Issue and merge request sidebar: - - | Sidebar| Collapsed sidebar | - | --- | --- | - | ![Busy status - sidebar](img/busy_indicator_sidebar_v13_9.png) | ![Busy status - sidebar collapsed](img/busy_indicator_sidebar_collapsed_v13_9.png) | - - Notes: - - | Notes | Note headers | - | --- | --- | - | ![Busy status - notes](img/busy_indicator_notes_v13_9.png) | ![Busy status - note header](img/busy_indicator_note_header_v13_9.png) | + The busy status is displayed next to your name, every time your name is shown in the user interface. ## Set your time zone @@ -310,9 +298,9 @@ and configure it on your local machine by using the following command: git config --global user.email <your email address> ``` -## User activity +## Follow users -GitLab tracks [user contribution activity](contributions_calendar.md). You can follow or unfollow other users from either: +You can follow or unfollow users from either: - Their [user profiles](#access-your-user-profile). - The small popover that appears when you hover over a user's name ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76050) @@ -321,66 +309,77 @@ GitLab tracks [user contribution activity](contributions_calendar.md). You can f In [GitLab 15.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/360755), the maximum number of users you can follow is 300. -To view a user's activity in a top-level Activity view: +### Disable following and being followed by other users + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325558) in GitLab 16.0 [with a flag](../feature_flags.md) named `disable_follow_users`. Disabled by default. + +You can disable following and being followed by other users. + +1. On the top bar, in the upper-right corner, select your avatar. +1. Select **Edit profile**. +1. Select **Preferences**. +1. Clear the **Enable follow users** checkbox. +1. Select **Save changes**. + +NOTE: +When this feature is being disabled, all current followed/following connections are deleted. + +## View your activity + +GitLab tracks [user contribution activity](contributions_calendar.md). +To view a summary of your activity, or the activity of other users: 1. From a user's profile, select **Follow**. 1. In the GitLab menu, select **Activity**. 1. Select the **Followed users** tab. -## Troubleshooting +## Session duration -### Why do you keep getting signed out? +### Stay signed in for two weeks -When you sign in to the main GitLab application, a `_gitlab_session` cookie is -set. When you close your browser, the cookie is cleared client-side -and it expires after a set duration. GitLab administrators can determine the duration: +By default, you are signed out of GitLab after seven days (10080 minutes) of inactivity or until you close your browser +window, whichever comes first. -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand **Account and limit**. The set duration is in **Session duration (minutes)**. +GitLab administrators can +[change this default](../admin_area/settings/account_and_limit_settings.md#customize-the-default-session-duration). -The default is `10080`, which equals 7 days. +### Stay signed in indefinitely -When you sign in to the main GitLab application, you can also check the -**Remember me** option. This sets the `remember_user_token` -cookie via [`devise`](https://github.com/heartcombo/devise). -The `remember_user_token` cookie expires after -`config/initializers/devise.rb` -> `config.remember_for`. The default is 2 weeks. +> Ability to turn the **Remember me** setting on and off [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/369133) in GitLab 16.0. -When the `_gitlab_session` expires or isn't available, GitLab uses the `remember_user_token` -to get you a new `_gitlab_session` and keep you signed in through browser restarts. +To remain signed in indefinitely, select the **Remember me** checkbox on the GitLab sign-in page. -After your `remember_user_token` expires and your `_gitlab_session` is cleared/expired, -you are asked to sign in again to verify your identity for security reasons. +You remain signed in because, although the server sets a session time of one week, your browser stores a secure token +that enables automatic reauthentication. -NOTE: -When any session is signed out, or when a session is revoked -via [Active Sessions](active_sessions.md), all **Remember me** tokens are revoked. -While other sessions remain active, the **Remember me** feature doesn't restore -a session if the browser is closed or the existing session expires. +GitLab administrators can [turn off the **Remember me** setting](../admin_area/settings/account_and_limit_settings.md#session-duration) for environments +that require sessions to expire periodically for security or compliance purposes. -### Increased sign-in time +### Cookies used for sign-in > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20340) in GitLab 13.1. -The `remember_user_token` lifetime of a cookie can now extend beyond the deadline set by `config.remember_for`, as the `config.extend_remember_period` flag is now set to true. +When you sign in, three cookies are set: -GitLab uses both session and persistent cookies: +- A session cookie called `_gitlab_session`. + This cookie has no set expiration date. However, it expires based on its `session_expire_delay`. +- A session cookied called `about_gitlab_active_user`. + This cookie is used by the [marketing site](https://about.gitlab.com/) to determine if a user has an active GitLab session. No user information is passed to the cookie and it expires with the session. +- A persistent cookie called `remember_user_token`, which is set only if you selected **Remember me** on the sign-in page. -- Session cookie: Session cookies are typically removed at the end of the browser session when - the browser is closed. The `_gitlab_session` cookie has no fixed expiration date. However, - it expires based on its [`session_expire_delay`](#why-do-you-keep-getting-signed-out). -- Persistent cookie: The `remember_user_token` is a cookie with an expiration date of two weeks. - GitLab activates this cookie if you select **Remember Me** when you sign in. +When you close your browser, the `_gitlab_session` and `about_gitlab_active_user` cookies are usually cleared client-side. +When it expires or isn't available, GitLab: -By default, the server sets a time-to-live (TTL) of 1-week on any session that is used. +- Uses the `remember_user_token`cookie to get you a new `_gitlab_session` cookie and keep you signed in, even if you close your browser. +- Sets the `about_gitlab_active_user` to `true`. -When you close a browser, the session cookie may still remain. For example, Chrome has the "Continue where you left off" option that restores session cookies. -In other words, as long as you access GitLab at least once every 2 weeks, you could remain signed in with GitLab, as long as your browser tab is open. -The server continues to reset the TTL for that session, independent of whether 2FA is installed, -If you close your browser and open it up again, the `remember_user_token` cookie allows your user to reauthenticate itself. +When both the `remember_user_token` and `_gitlab_session` cookies are gone or expired, you must sign in again. -Without the `config.extend_remember_period` flag, you would be forced to sign in again after two weeks. +NOTE: +When any session is signed out, or when a session is revoked +from the [active sessions list](active_sessions.md), all **Remember me** tokens are revoked. +While other sessions remain active, the **Remember me** feature doesn't restore +a session if the browser is closed or the existing session expires. ## Related topics @@ -389,7 +388,7 @@ Without the `config.extend_remember_period` flag, you would be forced to sign in - [Change your password](user_passwords.md) - Receive emails for: - [Sign-ins from unknown IP addresses or devices](notifications.md#notifications-for-unknown-sign-ins) - - [Attempted sign-ins using wrong two-factor authentication codes](notifications.md#notifications-for-attempted-sign-in-using-wrong-two-factor-authentication-codes) + - [Attempted sign-ins using incorrect verification codes](notifications.md#notifications-for-attempted-sign-ins-using-incorrect-verification-codes) - Manage applications that can [use GitLab as an OAuth provider](../../integration/oauth_provider.md) - Manage [personal access tokens](personal_access_tokens.md) to access your account via API and authorized applications - Manage [SSH keys](../ssh.md) to access your account via SSH diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index dcc78dac05b..b60ff8471bf 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/notifications.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments @@ -240,7 +239,7 @@ Turning this toggle off only unsubscribes you from updates related to this issue Learn how to [opt out of all emails from GitLab](#opt-out-of-all-gitlab-emails). <!-- Delete when the `moved_mr_sidebar` feature flag is removed --> -If you don't see this action on the right sidebar, your project or instance may have +If you don't see this action on the right sidebar, your project or instance might have enabled a feature flag for [moved sidebar actions](../project/merge_requests/index.md#move-sidebar-actions). ### Notification events on issues, merge requests, and epics @@ -286,7 +285,8 @@ To always receive notifications on your own issues, merge requests, and so on, t ## Notifications for unknown sign-ins -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27211) in GitLab 13.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27211) in GitLab 13.0. +> - Listing the full name and username of the signed-in user [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225183) in GitLab 15.10. NOTE: This feature is enabled by default for self-managed instances. Administrators may disable this feature @@ -295,7 +295,12 @@ The feature is always enabled on GitLab.com. When a user successfully signs in from a previously unknown IP address or device, GitLab notifies the user by email. In this way, GitLab proactively alerts users of potentially -malicious or unauthorized sign-ins. +malicious or unauthorized sign-ins. This notification email includes the: + +- Hostname. +- User's name and username. +- IP address. +- Date and time of sign-in. GitLab uses several methods to identify a known sign-in. All methods must fail for a notification email to be sent. @@ -306,7 +311,7 @@ GitLab uses several methods to identify a known sign-in. All methods must fail f - Cookie: After successful sign in, an encrypted cookie is stored in the browser. This cookie is set to expire 14 days after the last successful sign in. -## Notifications for attempted sign-in using wrong two-factor authentication codes +## Notifications for attempted sign-ins using incorrect verification codes > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/374740) in GitLab 15.5. @@ -357,6 +362,7 @@ The following table lists all GitLab-specific email headers: | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `List-Id` | The path of the project in an RFC 2919 mailing list identifier. You can use it for email organization with filters. | | `X-GitLab-(Resource)-ID` | The ID of the resource the notification is for. The resource, for example, can be `Issue`, `MergeRequest`, `Commit`, or another such resource. | +| `X-GitLab-ConfidentialIssue` | The boolean value indicating issue confidentiality for notifications. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222908) in GitLab 16.0. | | `X-GitLab-Discussion-ID` | The ID of the thread the comment belongs to, in notification emails for comments. | | `X-GitLab-Group-Id` | The group's ID. Only present on notification emails for [epics](../group/epics/index.md). | | `X-GitLab-Group-Path` | The group's path. Only present on notification emails for [epics](../group/epics/index.md) | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 3826c602fb4..e59d7313281 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -20,11 +20,7 @@ Personal access tokens can be an alternative to [OAuth2](../../api/oauth2.md) an In both cases, you authenticate with a personal access token in place of your password. WARNING: -The ability to create personal access tokens without expiry was -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab -16.0. When this ability is removed, existing personal access tokens without an expiry are planned to have an expiry added. -The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry -occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. +The ability to create personal access tokens without expiry was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. In GitLab 16.0 and later, existing personal access tokens without an expiry date are automatically given an expiry date of 365 days later than the current date. The automatic adding of an expiry date occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry date occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. Personal access tokens are: @@ -45,19 +41,20 @@ For examples of how you can use a personal access token to authenticate with the Alternately, GitLab administrators can use the API to create [impersonation tokens](../../api/rest/index.md#impersonation-tokens). Use impersonation tokens to automate authentication as a specific user. -NOTE: -Personal access tokens are not FIPS compliant and creation and use are disabled when [FIPS mode](../../development/fips_compliance.md) is enabled. - ## Create a personal access token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days is populated in the UI. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days is populated in the UI. +> - Ability to create non-expiring personal access tokens [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. You can create as many personal access tokens as you like. 1. In the upper-right corner, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Access Tokens**. -1. Enter a name and optional expiry date for the token. +1. Enter a name and expiry date for the token. + - The token expires on that date at midnight UTC. + - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. + - By default, this date can be a maximum of 365 days later than the current date. 1. Select the [desired scopes](#personal-access-token-scopes). 1. Select **Create personal access token**. @@ -103,6 +100,8 @@ To view the last time a token was used: ## Personal access token scopes +> Personal access tokens no longer being able to access container or package registries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387721) in GitLab 16.0. + A personal access token can perform actions based on the assigned scopes. | Scope | Access | @@ -117,6 +116,9 @@ A personal access token can perform actions based on the assigned scopes. | `sudo` | Grants permission to perform API actions as any user in the system, when authenticated as an administrator. | | `admin_mode` | Grants permission to perform API actions as an administrator, when Admin Mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107875) in GitLab 15.8.) | +WARNING: +If you enabled [external authorization](../admin_area/settings/external_authorization.md), personal access tokens cannot access container or package registries. If you use personal access tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use personal access tokens with container or package registries. + ## When personal access tokens expire Personal access tokens expire on the date you define, at midnight UTC. @@ -225,7 +227,7 @@ Remember this if you set up an automation pipeline that depends on authenticatio ### Unrevoke a personal access token **(FREE SELF)** -If a personal access token is revoked accidentally by any method, administrators can unrevoke that token. +If a personal access token is revoked accidentally by any method, administrators can unrevoke that token. By default, a daily job deletes revoked tokens at 1:00 AM system time. WARNING: Running the following commands changes data directly. This could be damaging if not done correctly, or under the right conditions. You should first run these commands in a test environment with a backup of the instance ready to be restored, just in case. @@ -247,5 +249,4 @@ Running the following commands changes data directly. This could be damaging if ## Alternatives to personal access tokens -For Git over HTTPS, an alternative to personal access tokens is [Git Credential Manager](account/two_factor_authentication.md#git-credential-manager), -which securely authenticates using OAuth. +For Git over HTTPS, an alternative to personal access tokens is to use an [OAuth credential helper](account/two_factor_authentication.md#oauth-credential-helpers). diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 7e581bb7419..da4d2da70fe 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -39,11 +39,11 @@ The default theme is Indigo. You can choose between 10 themes: ## Dark mode -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an [Alpha](../../policy/alpha-beta-support.md#alpha-features) release. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an [Experiment](../../policy/alpha-beta-support.md#experiment) release. -GitLab has started work on dark mode! The dark mode Alpha release is available in the +GitLab has started work on dark mode! The dark mode Experiment release is available in the spirit of iteration and the lower expectations of -[Alpha versions](../../policy/alpha-beta-support.md#alpha-features). +[Experiment features](../../policy/alpha-beta-support.md#experiment). Progress on dark mode is tracked in the [Dark theme epic](https://gitlab.com/groups/gitlab-org/-/epics/2902). See the epic for: @@ -63,7 +63,9 @@ Dark theme only works with the **Dark** syntax highlighting theme. ## Syntax highlighting theme -GitLab uses the [rouge Ruby library](http://rouge.jneen.net/ "Rouge website") +> Changing the default syntax highlighting theme for new users and users who are not signed in [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25129) in GitLab 15.10. + +GitLab uses the [Rouge Ruby library](https://github.com/rouge-ruby/rouge) for syntax highlighting outside of any Editor context. The WebIDE (like Snippets) uses [Monaco Editor](https://microsoft.github.io/monaco-editor/) and it's provided [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for @@ -73,22 +75,14 @@ the respective libraries. Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. -The default syntax theme is White, and you can choose among 5 different themes: - -<!-- vale gitlab.Spelling = NO --> - -- White -- Dark -- Solarized light -- Solarized dark -- Monokai - -<!-- vale gitlab.Spelling = YES --> - -![Profile preferences syntax highlighting themes](img/profile-preferences-syntax-themes.png) +![Profile preferences syntax highlighting themes](img/profile-preferences-syntax-themes_v15_11.png) Introduced in GitLab 13.6, the themes [Solarized](https://gitlab.com/gitlab-org/gitlab/-/issues/221034) and [Monokai](https://gitlab.com/gitlab-org/gitlab/-/issues/221034) also apply to the [Web IDE](../project/web_ide/index.md) and [Snippets](../snippets.md). +You can use an API call to change the default syntax highlighting theme for new users and users +who are not signed in. For more information, see the `default_syntax_highlighting_theme` +in the [list of settings that can be accessed through API calls](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). + ## Diff colors A diff compares the old/removed content with the new/added content (for example, when @@ -116,12 +110,9 @@ between the fixed (max. `1280px`) and the fluid (`100%`) application layout. NOTE: While `1280px` is the standard max width when using fixed layout, some pages still use 100% width, depending on the content. -### Dashboard +### Homepage -For users who have access to a large number of projects but only keep up with a -select few, the amount of activity on the your dashboard can be -overwhelming. From the **Dashboard** dropdown list, select what you'd like displayed on your -personal dashboard. +This setting changes the behavior of the tanuki icon in the upper-left corner of GitLab. ### Group overview content @@ -191,22 +182,6 @@ NOTE: This feature is experimental, and choosing absolute times might break certain layouts. Open an issue if you notice that using absolute times breaks a layout. -## Web IDE - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370139) in GitLab 15.7 [with a flag](../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. - -The [Web IDE Beta](../project/web_ide_beta/index.md) is -the default editing environment when the `vscode_web_ide` feature -flag is enabled. - -To stop using the Web IDE Beta: - -1. In the **Web IDE** section, select the **Opt out of the Web IDE Beta** checkbox. -1. Select **Save changes**. - ## Integrations Configure your preferences with third-party services which provide enhancements to your GitLab experience. diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/profile/saved_replies.md index 4125d8e8fdb..1f4e4f5fa51 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/profile/saved_replies.md @@ -1,11 +1,11 @@ --- -redirect_to: 'index.md' -remove_date: '2023-03-31' +redirect_to: 'comment_templates.md' +remove_date: '2023-06-22' --- -This document was moved to [another location](index.md). +This document was moved to [another location](comment_templates.md). -<!-- This redirect file can be deleted after <2023-03-31>. --> +<!-- This redirect file can be deleted after <2023-06-22>. --> <!-- Redirects that point to other docs in the same project expire in three months. --> <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md index 0ea1a80bc54..26708dece50 100644 --- a/doc/user/project/badges.md +++ b/doc/user/project/badges.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -62,7 +62,7 @@ You can access a test coverage report badge image by using the following link: https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg ``` -You can define the regular expression for the [coverage report](../../ci/pipelines/settings.md#merge-request-test-coverage-results) +You can define the regular expression for the [coverage report](../../ci/testing/code_coverage.md#view-code-coverage-results-in-the-mr) that each job log is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. @@ -107,7 +107,7 @@ If there is no release, it shows `none`. You can access a latest release badge image by using the following link: ```plaintext -https://gitlab.example.com/<namespace>/<project>/badges/<branch>/release.svg +https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg ``` By default, the badge fetches the release sorted using the [`released_at`](../../api/releases/index.md#create-a-release) @@ -117,6 +117,10 @@ time with the `?order_by` query parameter. https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg?order_by=release_at ``` +You can change the width of the release name field by using the `value_width` parameter ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113615) in GitLab 15.10). +The value must be between 1 and 200, and the default value is 54. +If you set an out of range value, GitLab automatically adjusts it to the default value. + ## Project badges Badges can be added to a project by Maintainers or Owners, and are visible on the project's overview page. @@ -275,7 +279,7 @@ To add a new badge with a custom image to a group or project: 1. Select **Add badge**. To learn how to use custom images generated through a pipeline, see the documentation on -[accessing the latest job artifacts by URL](../../ci/pipelines/job_artifacts.md#access-the-latest-job-artifacts). +[accessing the latest job artifacts by URL](../../ci/jobs/job_artifacts.md#from-a-url). ## Placeholders diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 95f38c7e354..8ea762777ac 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -4,18 +4,16 @@ group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Canary Deployments **(FREE)** +# Canary deployments **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1659) in GitLab 9.1. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. -A popular [Continuous Deployment](https://en.wikipedia.org/wiki/Continuous_deployment) +Canary deployments are a popular [continuous deployment](https://en.wikipedia.org/wiki/Continuous_deployment) strategy, where a small portion of the fleet is updated to the new version of your application. -## Overview - -When embracing [Continuous Delivery](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/), a company needs to decide what +When embracing [continuous delivery](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/), an organization needs to decide what type of deployment strategy to use. One of the most popular strategies is canary deployments, where a small portion of the fleet is updated to the new version first. This subset, the canaries, then serve as the proverbial diff --git a/doc/user/project/changelogs.md b/doc/user/project/changelogs.md index a64edd053b1..7e622110291 100644 --- a/doc/user/project/changelogs.md +++ b/doc/user/project/changelogs.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, api --- -# Changelogs +# Changelogs **(FREE)** Changelogs are generated based on commit titles and Git trailers. To be included in a changelog, a commit must contain a specific Git trailer. Changelogs are generated @@ -313,5 +313,5 @@ an error is produced when generating a changelog. ## Related topics -- [Changelog-related endpoints](../../api/repositories.md) in the Repositories API. -- Developer documentation for [changelog entries](../../development/changelog.md) in GitLab. +- [Changelog-related endpoints](../../api/repositories.md) in the Repositories API +- Developer documentation for [changelog entries](../../development/changelog.md) in GitLab diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 52288af101a..6c8edb8c3e5 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Connect EKS clusters through cluster certificates (DEPRECATED) **(FREE)** +# Connect EKS clusters through cluster certificates (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22392) in GitLab 12.5. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. @@ -25,7 +25,7 @@ use the [GitLab agent](../../clusters/agent/index.md). To create a new cluster from GitLab, use [Infrastructure as Code](../../infrastructure/iac/index.md). -### How to create a new cluster on EKS through cluster certificates (DEPRECATED) +### How to create a new cluster on EKS through cluster certificates (deprecated) > [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. @@ -174,11 +174,11 @@ When you create a new cluster, you have the following settings: | Kubernetes cluster name | Your cluster's name. | | Environment scope | The [associated environment](multiple_kubernetes_clusters.md#setting-the-environment-scope). | | Service role | The **EKS IAM role** (**role A**). | -| Kubernetes version | The [Kubernetes version](../../clusters/agent/index.md#supported-cluster-versions) for your cluster. | +| Kubernetes version | The [Kubernetes version](../../clusters/agent/index.md#gitlab-agent-for-kubernetes-supported-cluster-versions) for your cluster. | | Key pair name | The [key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html) that you can use to connect to your worker nodes. | | VPC | The [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) to use for your EKS Cluster resources. | | Subnets | The [subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) in your VPC where your worker nodes run. Two are required. | -| Security group | The [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) to apply to the EKS-managed Elastic Network Interfaces that are created in your worker node subnets. | +| Security group | The [security group](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-security-groups.html) to apply to the EKS-managed Elastic Network Interfaces that are created in your worker node subnets. | | Instance type | The [instance type](https://aws.amazon.com/ec2/instance-types/) of your worker nodes. | | Node count | The number of worker nodes. | | GitLab-managed cluster | Check if you want GitLab to manage namespaces and service accounts for this cluster. | diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index 0b0555806ce..bb85662d442 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Connect existing clusters through cluster certificates (DEPRECATED) **(FREE)** +# Connect existing clusters through cluster certificates (deprecated) **(FREE)** > [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index b1c5bbfc4f7..c6561fffa0b 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Connect GKE clusters through cluster certificates (DEPRECATED) **(FREE)** +# Connect GKE clusters through cluster certificates (deprecated) **(FREE)** > [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. @@ -77,7 +77,7 @@ cluster certificates: - **Zone** - Choose the [region zone](https://cloud.google.com/compute/docs/regions-zones/) under which to create the cluster. - **Number of nodes** - Enter the number of nodes you wish the cluster to have. - - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) + - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-resource) of the Virtual Machine instance to base the cluster on. - **Enable Cloud Run for Anthos** - Check this if you want to use Cloud Run for Anthos for this cluster. See the [Cloud Run for Anthos section](#cloud-run-for-anthos) for more information. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index d3048158958..bba05f1e724 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Add a cluster using cluster certificates (DEPRECATED) **(FREE)** +# Add a cluster using cluster certificates (deprecated) **(FREE)** > [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 529e7a6da12..ff30da2ca98 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Access controls with cluster certificates (RBAC or ABAC) (DEPRECATED) **(FREE)** +# Access controls with cluster certificates (RBAC or ABAC) (deprecated) **(FREE)** > - Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index 173f1f39a66..31d5a73757a 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Deploy to a Kubernetes cluster with cluster certificates (DEPRECATED) **(FREE)** +# Deploy to a Kubernetes cluster with cluster certificates (deprecated) **(FREE)** > [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index b2a4bd85de4..b321646d8d8 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab-managed clusters (DEPRECATED) **(FREE)** +# GitLab-managed clusters (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. > - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 94513fc7124..5ce1994ba36 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Project-level Kubernetes clusters (certificate-based) (DEPRECATED) **(FREE)** +# Project-level Kubernetes clusters (certificate-based) (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/35954) in GitLab 10.1. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. diff --git a/doc/user/project/clusters/multiple_kubernetes_clusters.md b/doc/user/project/clusters/multiple_kubernetes_clusters.md index c79f250da7a..c8f49b1917d 100644 --- a/doc/user/project/clusters/multiple_kubernetes_clusters.md +++ b/doc/user/project/clusters/multiple_kubernetes_clusters.md @@ -1,10 +1,10 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Multiple clusters per project with cluster certificates (DEPRECATED) **(FREE)** +# Multiple clusters per project with cluster certificates (deprecated) **(FREE)** > - Introduced in GitLab 10.3 > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) from GitLab Premium to GitLab Free in 13.2. diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index df0ffff8561..a985436d67d 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -107,7 +107,7 @@ the components outlined above and the pre-loaded demo runbook. ''' We set user's id, login and access token on single user image to enable repository integration for JupyterHub. - See: https://gitlab.com/gitlab-org/gitlab-foss/issues/47138#note_154294790 + See: https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47138#note_154294790 ''' auth_state = await spawner.user.get_auth_state() diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 9de9d445965..f9244177aa5 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -1,373 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'codeowners/index.md' +remove_date: '2023-07-07' --- -# Code Owners **(PREMIUM)** +This document was moved to [another location](codeowners/index.md). -> Moved to GitLab Premium in 13.9. - -Code Owners define who develops and maintains a feature, and own the resulting -files or directories in a repository. - -- The users you define as Code Owners are displayed in the UI when you browse directories. -- You can set your merge requests so they must be approved by Code Owners before merge. -- You can protect a branch and allow only Code Owners to approve changes to the branch. - -<div class="video-fallback"> - Video introduction: <a href="https://www.youtube.com/watch?v=RoyBySTUSB0">Code Owners</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/RoyBySTUSB0" frameborder="0" allowfullscreen> </iframe> -</figure> - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> - -Use Code Owners and approvers together with -[approval rules](merge_requests/approvals/rules.md) to build a flexible approval -workflow: - -- Use **Code Owners** to define the users who have domain expertise for specific paths in your repository. -- Use **Approvers** and **Approval rules** to define domains of expertise (such as a security team) - that are not scoped to specific file paths in your repository. - - **Approvers** define the users. - - **Approval rules** define when these users can approve work, and whether or not their approval is required. - -For example: - -| Type | Name | Scope | Comment | -|------|------|--------|------------| -| Approval rule | UX | All files | A user experience (UX) team member reviews the user experience of all changes made in your project. | -| Approval rule | Security | All files | A security team member reviews all changes for vulnerabilities. | -| Code Owner approval rule | Frontend: Code Style | `*.css` files | A frontend engineer reviews CSS file changes for adherence to project style standards. | -| Code Owner approval rule | Backend: Code Review | `*.rb` files | A backend engineer reviews the logic and code style of Ruby files. | - -## Code Owners file - -A `CODEOWNERS` file (with no extension) can specify users or [shared groups](members/share_project_with_groups.md) -that are responsible for specific files and directories in a repository. Each repository -can have a single `CODEOWNERS` file, and it must be found one of these three locations: - -- In the root directory of the repository. -- In the `.gitlab/` directory. -- In the `docs/` directory. - -A CODEOWNERS file in any other location is ignored. - -## Set up Code Owners - -1. Create a file named `CODEOWNERS` (with no extension) in one of these locations: - -- In the root directory of the repository -- In the `.gitlab/` directory -- In the `docs/` directory - -1. In the file, enter text that follows one of these patterns: - - ```plaintext - # Code Owners for a file - filename @username1 @username2 - - # Code Owners for a directory - directoryname/ @username1 @username2 - - # All group members as Code Owners for a file - filename @groupname - - # All group members as Code Owners for a directory - directoryname/ @groupname - ``` - -The Code Owners are now displayed in the UI. They apply to the current branch only. - -Next steps: - -- [Add Code Owners as merge request approvers](merge_requests/approvals/rules.md#code-owners-as-eligible-approvers). -- Set up [Code Owner approval on a protected branch](protected_branches.md#require-code-owner-approval-on-a-protected-branch). - -## Groups as Code Owners - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53182) in GitLab 12.1. -> - Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. - -You can use members of groups and subgroups as Code Owners for projects: - -```mermaid -graph TD - A[Parent group X] -->|owns| B[Project A] - A -->|contains| C[Subgroup Y] - C -->|owns| D[Project B] - A-. inherits ownership .-> D -``` - -In this example: - -- **Parent group X** (`group-x`) owns **Project A**. -- **Parent group X** also contains a subgroup, **Subgroup Y**. (`group-x/subgroup-y`) -- **Subgroup Y** owns **Project B**. - -The eligible Code Owners are: - -- **Project A**: the members of **Group X** only, because **Project A** doesn't belong to **Subgroup Y**. -- **Project B**: the members of both **Group X** and **Subgroup Y**. - -You can [invite](members/share_project_with_groups.md) **Subgroup Y** to **Project A** -so that their members also become eligible Code Owners. - -```mermaid -graph LR - A[Parent group X] -->|owns| B[Project A] - A -->|also contains| C[Subgroup Y] - C -.->D{Invite Subgroup Y<br/>to Project A?} -.->|yes| F[Approvals can be<br/> required] -.-> B - D{Invite Subgroup Y<br/>to Project A?} -.->|no| I[Subgroup Y cannot be<br /> an approver] -.-> B - C -.->E{Add Subgroup Y<br/>as Code Owners<br/>to Project A?} -.->|yes| H[Approvals can only<br/>be optional] -.-> B -``` - -If you do not invite **Subgroup Y** to **Project A**, but make them Code Owners, their approval -of the merge request becomes optional. - -Inviting **Subgroup Y** to a parent group of **Project A** -[is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/288851). To set **Subgroup Y** as -Code Owners, add this group directly to the project itself. - -NOTE: -For approval to be required, groups as Code Owners must have a direct membership -(not inherited membership) in the project. Approval can only be optional for groups -that inherit membership. Members in the Code Owners group also must be direct members, -and not inherit membership from any parent groups. - -### Add a group as a Code Owner - -To set a group as a Code Owner: - -In the `CODEOWNERS` file, enter text that follows one of these patterns: - -```plaintext -# All group members as Code Owners for a file -file.md @group-x - -# All subgroup members as Code Owners for a file -file.md @group-x/subgroup-y - -# All group and subgroup members as Code Owners for a file -file.md @group-x @group-x/subgroup-y -``` - -## When a file matches multiple `CODEOWNERS` entries - -When a file matches multiple entries in the `CODEOWNERS` file, -the users from last pattern matching the file are used. - -For example, in the following `CODEOWNERS` file: - -```plaintext -README.md @user1 - -# This line would also match the file README.md -*.md @user2 -``` - -The Code Owner for `README.md` would be `@user2`. - -If you use sections, the last pattern matching the file for each section is used. -For example, in a `CODEOWNERS` file using sections: - -```plaintext -[README Owners] -README.md @user1 @user2 -internal/README.md @user4 - -[README other owners] -README.md @user3 -``` - -The Code Owners for the `README.md` in the root directory are `@user1`, `@user2`, -and `@user3`. The Code Owners for `internal/README.md` are `@user4` and `@user3`. - -Only one CODEOWNERS pattern can match per file path. - -### Organize Code Owners by putting them into sections - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab 13.2 [with a flag](../../administration/feature_flags.md) named `sectional_codeowners`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Feature flag `sectional_codeowners` removed. - -You can organize Code Owners by putting them into named sections. - -You can use sections for shared directories, so that multiple -teams can be reviewers. - -To add a section to the `CODEOWNERS` file, enter a section name in brackets, -followed by the files or directories, and users, groups, or subgroups: - -```plaintext -[README Owners] -README.md @user1 @user2 -internal/README.md @user2 -``` - -Each Code Owner in the merge request widget is listed under a label. -The following image shows a **Groups** and **Documentation** section: - -![MR widget - Sectional Code Owners](img/sectional_code_owners_v13.2.png) - -### Sections with duplicate names - -If multiple sections have the same name, they are combined. -Also, section headings are not case-sensitive. For example: - -```plaintext -[Documentation] -ee/docs/ @docs -docs/ @docs - -[Database] -README.md @database -model/db/ @database - -[DOCUMENTATION] -README.md @docs -``` - -This code results in three entries under the **Documentation** section header, and two -entries under **Database**. The entries defined under the sections **Documentation** and -**DOCUMENTATION** are combined, using the case of the first section. - -### Make a Code Owners section optional - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab 13.8. - -You can designate optional sections in your Code Owners file. Prepend the -section name with the caret `^` character to treat the entire section as optional. -Optional sections enable you to designate responsible parties for various parts -of your codebase, but not require approval from them. This approach provides -a more relaxed policy for parts of your project that are frequently updated, -but don't require stringent reviews. - -In this example, the `[Go]` section is optional: - -```plaintext -[Documentation] -*.md @root - -[Ruby] -*.rb @root - -^[Go] -*.go @root -``` - -The optional Code Owners section displays in merge requests under the **Approval Rules** area: - -![MR widget - Optional Code Owners sections](img/optional_code_owners_sections_v13_8.png) - -If a section is duplicated in the file, and one of them is marked as optional and the other isn't, the section is required. - -Optional sections in the `CODEOWNERS` file are treated as optional only -when changes are submitted by using merge requests. If a change is submitted directly -to the protected branch, approval from Code Owners is still required, even if the -section is marked as optional. - -### Allowed to Push - -The Code Owner approval and protected branch features do not apply to users who -are **Allowed to push**. - -## Example `CODEOWNERS` file - -```plaintext -# This is an example of a CODEOWNERS file. -# Lines that start with `#` are ignored. - -# app/ @commented-rule - -# Specify a default Code Owner by using a wildcard: -* @default-codeowner - -# Specify multiple Code Owners by using a tab or space: -* @multiple @code @owners - -# Rules defined later in the file take precedence over the rules -# defined before. -# For example, for all files with a filename ending in `.rb`: -*.rb @ruby-owner - -# Files with a `#` can still be accessed by escaping the pound sign: -\#file_with_pound.rb @owner-file-with-pound - -# Specify multiple Code Owners separated by spaces or tabs. -# In the following case the CODEOWNERS file from the root of the repo -# has 3 Code Owners (@multiple @code @owners): -CODEOWNERS @multiple @code @owners - -# You can use both usernames or email addresses to match -# users. Everything else is ignored. For example, this code -# specifies the `@legal` and a user with email `janedoe@gitlab.com` as the -# owner for the LICENSE file: -LICENSE @legal this_does_not_match janedoe@gitlab.com - -# Use group names to match groups, and nested groups to specify -# them as owners for a file: -README @group @group/with-nested/subgroup - -# End a path in a `/` to specify the Code Owners for every file -# nested in that directory, on any level: -/docs/ @all-docs - -# End a path in `/*` to specify Code Owners for every file in -# a directory, but not nested deeper. This code matches -# `docs/index.md` but not `docs/projects/index.md`: -/docs/* @root-docs - -# Include `/**` to specify Code Owners for all subdirectories -# in a directory. This rule matches `docs/projects/index.md` or -# `docs/development/index.md` -/docs/**/*.md @root-docs - -# This code makes matches a `lib` directory nested anywhere in the repository: -lib/ @lib-owner - -# This code match only a `config` directory in the root of the repository: -/config/ @config-owner - -# If the path contains spaces, escape them like this: -path\ with\ spaces/ @space-owner - -# Code Owners section: -[Documentation] -ee/docs @docs -docs @docs - -[Database] -README.md @database -model/db @database - -# This section is combined with the previously defined [Documentation] section: -[DOCUMENTATION] -README.md @docs -``` - -## Troubleshooting - -### Approvals shown as optional - -A Code Owner approval rule is optional if any of these conditions are true: - -- The user or group are not a member of the project. Code Owners [cannot inherit from parent groups](https://gitlab.com/gitlab-org/gitlab/-/issues/288851/). -- [Code Owner approval on a protected branch](protected_branches.md#require-code-owner-approval-on-a-protected-branch) has not been set up. -- The section is [marked as optional](#make-a-code-owners-section-optional). - -### Approvals do not show - -Code Owner approval rules only update when the merge request is created. -If you update the `CODEOWNERS` file, close the merge request and create a new one. - -### User not shown as possible approver - -A user might not show as an approver on the Code Owner merge request approval rules -if any of these conditions are true: - -- A rule prevents the specific user from approving the merge request. - Check the project [merge request approval](merge_requests/approvals/settings.md#edit-merge-request-approval-settings) settings. -- A Code Owner group has a visibility of **private**, and the current user is not a - member of the Code Owner group. +<!-- This redirect file can be deleted after <2023-07-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/codeowners/index.md b/doc/user/project/codeowners/index.md new file mode 100644 index 00000000000..e69dba6f977 --- /dev/null +++ b/doc/user/project/codeowners/index.md @@ -0,0 +1,381 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code Owners **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +Use the Code Owners feature to define who has expertise for specific parts of your project's codebase. +Define the owners of files and directories in a repository to: + +- **Require owners to approve changes.** Combine protected branches with Code Owners to require + experts to approve merge requests before they merge into a protected branch. +- **Identify owners.** Code Owner names are displayed on the files and directories they own: + ![Code Owners displayed in UI](../img/codeowners_in_UI_v15_10.png) + +Use Code Owners in combination with merge request +[approval rules](../merge_requests/approvals/rules.md) (either optional or required) +to build a flexible approval workflow: + +- Use **Code Owners** to ensure quality. Define the users who have domain expertise + for specific paths in your repository. +- Use **Approval rules** to define areas of expertise that don't correspond to specific + file paths in your repository. Approval rules help guide merge request creators to + the correct set of reviewers, such as frontend developers or a security team. + +For example: + +| Type | Name | Scope | Comment | +|------|------|--------|------------| +| Approval rule | UX | All files | A user experience (UX) team member reviews the user experience of all changes made in your project. +| Approval rule | Security | All files | A security team member reviews all changes for vulnerabilities. +| Code Owner approval rule | Frontend: Code Style | `*.css` files | A frontend engineer reviews CSS file changes for adherence to project style standards. +| Code Owner approval rule | Backend: Code Review | `*.rb` files | A backend engineer reviews the logic and code style of Ruby files. + +<div class="video-fallback"> + Video introduction: <a href="https://www.youtube.com/watch?v=RoyBySTUSB0">Code Owners</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/RoyBySTUSB0" frameborder="0" allowfullscreen> </iframe> +</figure> + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> + +## View Code Owners of a file or directory + +To view the Code Owners of a file or directory: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files**. +1. Go to the file or directory you want to see the Code Owners for. +1. Optional. Select a branch or tag. + +GitLab shows the Code Owners at the top of the page. + +## Set up Code Owners + +1. Create a `CODEOWNERS` file in your [preferred location](#code-owners-file). +1. Define some rules in the file following the [Code Owners syntax reference](reference.md). + Some suggestions: + - Configure [All eligible approvers](../merge_requests/approvals/rules.md#code-owners-as-eligible-approvers) approval rule. + - [Require Code Owner approval](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) on a protected branch. +1. Commit your changes, and push them up to GitLab. + +### Code Owners file + +A `CODEOWNERS` file (with no extension) specifies the users or +[shared groups](../members/share_project_with_groups.md) responsible for +specific files and directories in a repository. + +Each repository uses a single `CODEOWNERS` file. GitLab checks these locations +in your repository in this order. The first `CODEOWNERS` file found is used, and +all others are ignored: + +1. In the root directory: `./CODEOWNERS`. +1. In the `docs` directory: `./docs/CODEOWNERS`. +1. In the `.gitlab` directory: `./.gitlab/CODEOWNERS`. + +### Add a group as a Code Owner + +> Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. + +To set the members of a group or subgroup as a Code Owner: + +In the `CODEOWNERS` file, enter text that follows one of these patterns: + +```plaintext +# All group members as Code Owners for a file +file.md @group-x + +# All subgroup members as Code Owners for a file +file.md @group-x/subgroup-y + +# All group and subgroup members as Code Owners for a file +file.md @group-x @group-x/subgroup-y +``` + +#### Group inheritance and eligibility + +> Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. + +```mermaid +graph TD + A[Parent group X] -->|owns| B[Project A] + A -->|contains| C[Subgroup Y] + C -->|owns| D[Project B] + A-. inherits ownership .-> D +``` + +In this example: + +- **Parent group X** (`group-x`) owns **Project A**. +- **Parent group X** also contains a subgroup, **Subgroup Y**. (`group-x/subgroup-y`) +- **Subgroup Y** owns **Project B**. + +The eligible Code Owners are: + +- **Project A**: the members of **Group X** only, because **Project A** doesn't belong to **Subgroup Y**. +- **Project B**: the members of both **Group X** and **Subgroup Y**. + +##### Inviting subgroups to projects in parent groups + +You can [invite](../members/share_project_with_groups.md) **Subgroup Y** to **Project A** +so that their members also become eligible Code Owners. + +```mermaid +graph LR + A[Parent group X] -->|owns| B[Project A] + A -->|also contains| C[Subgroup Y] + C -.->D{Invite Subgroup Y<br/>to Project A?} -.->|yes| E[Members of Subgroup Y<br/>can submit Approvals] + D{Invite Subgroup Y<br/>to Project A?} -.->|no| F[Members of Subgroup Y<br />cannot submit Approvals] + E -.->|Add Subgroup Y<br/> as Code Owner to Project A| I[Approvals can be<br/>required] -.-> B + F -.-> |Add Subgroup Y<br/> as Code Owners to Project A| J[Approvals can only<br/>be optional] -.-> B +``` + +If you do not invite **Subgroup Y** to **Project A**, but make them Code Owners, their approval +of the merge request becomes optional. + +##### Inviting subgroups to parent groups + +Inviting **Subgroup Y** to a parent group of **Project A** +[is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/288851). To set **Subgroup Y** as +Code Owners [invite this group directly to the project](#inviting-subgroups-to-projects-in-parent-groups) itself. + +NOTE: +For approval to be required, groups as Code Owners must have a direct membership +(not inherited membership) in the project. Approval can only be optional for groups +that inherit membership. Members in the Code Owners group also must be direct members, +and not inherit membership from any parent groups. + +### Define more specific owners for more specifically defined files or directories + +When a file or directory matches multiple entries in the `CODEOWNERS` file, +the users from last pattern matching the file or directory are used. This enables you +to define more specific owners for more specifically defined files or directories, when +you order the entries in a sensible way. + +For example, in the following `CODEOWNERS` file: + +```plaintext +# This line would match the file terms.md +*.md @doc-team + +# This line would also match the file terms.md +terms.md @legal-team +``` + +The Code Owner for `terms.md` would be `@legal-team`. + +If you use sections, the last pattern matching the file or directory for each section is used. +For example, in a `CODEOWNERS` file using sections: + +```plaintext +[README Owners] +README.md @user1 @user2 +internal/README.md @user4 + +[README other owners] +README.md @user3 +``` + +The Code Owners for the `README.md` in the root directory are `@user1`, `@user2`, +and `@user3`. The Code Owners for `internal/README.md` are `@user4` and `@user3`. + +Only one CODEOWNERS pattern per section is matched to a file path. + +### Organize Code Owners by putting them into sections + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab 13.2 [with a flag](../../../administration/feature_flags.md) named `sectional_codeowners`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Feature flag `sectional_codeowners` removed. + +You can organize Code Owners by putting them into named sections. + +You can use sections for shared directories, so that multiple +teams can be reviewers. + +To add a section to the `CODEOWNERS` file, enter a section name in brackets, +followed by the files or directories, and users, groups, or subgroups: + +```plaintext +[README Owners] +README.md @user1 @user2 +internal/README.md @user2 +``` + +Each Code Owner in the merge request widget is listed under a label. +The following image shows a **Groups** and **Documentation** section: + +![MR widget - Sectional Code Owners](../img/sectional_code_owners_v13.2.png) + +#### Set default owner for a section + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371711) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `codeowners_default_owners`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115888) in GitLab 15.11. Feature flag `codeowners_default_owners` removed. + +If multiple file paths inside a section share the same ownership, define a default +Code Owner for the section. All paths in that section inherit this default, unless +you override the section default on a specific line. + +Default owners are applied when specific owners are not specified for file paths. +Specific owners defined beside the file path override default owners: + +```plaintext +[Documentation] @docs-team +docs/ +README.md + +[Database] @database-team +model/db/ +config/db/database-setup.md @docs-team +``` + +In this example: + +- `@docs-team` owns all items in the `Documentation` section. +- `@database-team` owns all items in the `Database` section except + `config/db/database-setup.md`, which has an override assigning it to `@docs-team`. + +To combine the syntax for default owners with [optional sections](#make-a-code-owners-section-optional) +and required approvals, place default owners at the end: + +```plaintext +[Documentation][2] @docs-team +docs/ +README.md + +^[Database] @database-team +model/db/ +config/db/database-setup.md @docs-team +``` + +#### Sections with duplicate names + +If multiple sections have the same name, they are combined. +Also, section headings are not case-sensitive. For example: + +```plaintext +[Documentation] +ee/docs/ @docs +docs/ @docs + +[Database] +README.md @database +model/db/ @database + +[DOCUMENTATION] +README.md @docs +``` + +This code results in three entries under the **Documentation** section header, and two +entries under **Database**. The entries defined under the sections **Documentation** and +**DOCUMENTATION** are combined, using the case of the first section. + +#### Make a Code Owners section optional + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab 13.8. + +You can designate optional sections in your Code Owners file. Prepend the +section name with the caret `^` character to treat the entire section as optional. +Optional sections enable you to designate responsible parties for various parts +of your codebase, but not require approval from them. This approach provides +a more relaxed policy for parts of your project that are frequently updated, +but don't require stringent reviews. + +In this example, the `[Go]` section is optional: + +```plaintext +[Documentation] +*.md @root + +[Ruby] +*.rb @root + +^[Go] +*.go @root +``` + +The optional Code Owners section displays in merge requests under the **Approval Rules** area: + +![MR widget - Optional Code Owners sections](../img/optional_code_owners_sections_v13_8.png) + +If a section is duplicated in the file, and one of them is marked as optional and the other isn't, the section is required. + +Optional sections in the `CODEOWNERS` file are treated as optional only +when changes are submitted by using merge requests. If a change is submitted directly +to the protected branch, approval from Code Owners is still required, even if the +section is marked as optional. + +### Require multiple approvals from Code Owners + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335451) in GitLab 15.9. + +You can require multiple approvals for the Code Owners sections under the Approval Rules area in merge requests. +Append the section name with a number `n` in brackets. This requires `n` approvals from the Code Owners in this section. +Please note valid entries for `n` are integers `≥ 1`. `[1]` is optional as it is the default. Invalid values for `n` are treated as `1`. + +WARNING: +[Issue #384881](https://gitlab.com/gitlab-org/gitlab/-/issues/385881) proposes changes +to the behavior of this setting. Do not intentionally set invalid values. They may +become valid in the future, and cause unexpected behavior. + +Please confirm you enabled `Require approval from code owners` in `Settings > Repository > Protected branches`, otherwise the Code Owner approvals will be optional. + +In this example, the `[Documentation]` section requires 2 approvals: + +```plaintext +[Documentation][2] +*.md @tech-writer-team + +[Ruby] +*.rb @dev-team +``` + +The `Documentation` Code Owners section under the **Approval Rules** area displays 2 approvals are required: + +![MR widget - Multiple Approval Code Owners sections](../img/multi_approvals_code_owners_sections_v15_9.png) + +### Allowed to Push + +The Code Owner approval and protected branch features do not apply to users who +are **Allowed to push**. + +## Technical Resources + +[Code Owners development guidelines](../../../development/code_owners/index.md) + +## Troubleshooting + +For more information about how the Code Owners feature handles errors, see the +[Code Owners reference](reference.md). + +### Approvals shown as optional + +A Code Owner approval rule is optional if any of these conditions are true: + +- The user or group are not a member of the project. Code Owners [cannot inherit from parent groups](https://gitlab.com/gitlab-org/gitlab/-/issues/288851/). +- [Code Owner approval on a protected branch](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) has not been set up. +- The section is [marked as optional](#make-a-code-owners-section-optional). + +### Approvals do not show + +Code Owner approval rules only update when the merge request is created. +If you update the `CODEOWNERS` file, close the merge request and create a new one. + +### User not shown as possible approver + +A user might not show as an approver on the Code Owner merge request approval rules +if any of these conditions are true: + +- A rule prevents the specific user from approving the merge request. + Check the project [merge request approval](../merge_requests/approvals/settings.md#edit-merge-request-approval-settings) settings. +- A Code Owner group has a visibility of **private**, and the current user is not a + member of the Code Owner group. +- Current user is an external user who does not have permission to the internal Code Owner group. + +### Approval rule is invalid. GitLab has approved this rule automatically to unblock the merge request + +This message may appear if an approval rule uses a Code Owner that is not a direct member of the project. +Check that the group or user has been invited to the project. diff --git a/doc/user/project/codeowners/reference.md b/doc/user/project/codeowners/reference.md new file mode 100644 index 00000000000..d486b689638 --- /dev/null +++ b/doc/user/project/codeowners/reference.md @@ -0,0 +1,371 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code Owners syntax and error handling **(PREMIUM)** + +This page describes the syntax and error handling used in Code Owners files, +and provides an example file. + +## Code Owners syntax + +### Comments + +Lines beginning with `#` are ignored: + +```plaintext +# This is a comment +``` + +### Sections + +Sections are groups of entries. A section begins with a section heading in square brackets, followed by the entries. + +```plaintext +[Section name] +/path/of/protected/file.rb @username +/path/of/protected/dir/ @group +``` + +#### Section headings + +Section headings must always have a name. They can also be made optional, or +require a number of approvals. A list of default owners can be added to the section heading line. + +```plaintext +# Required section +[Section name] + +# Optional section +^[Section name] + +# Section requiring 5 approvals +[Section name][5] + +# Section with @username as default owner +[Section name] @username + +# Section with @group and @subgroup as default owners and requiring 2 approvals +[Section name][2] @group @subgroup +``` + +#### Section names + +Sections names are defined between square brackets. Section names are not case-sensitive. +[Sections with duplicate names](index.md#sections-with-duplicate-names) are combined. + +```plaintext +[Section name] +``` + +#### Required sections + +Required sections do not include `^` before the [section name](#section-names). + +```plaintext +[Required section] +``` + +#### Optional sections + +Optional sections include a `^` before the [section name](#section-names). + +```plaintext +^[Optional section] +``` + +#### Sections requiring multiple approvals + +Sections requiring multiple approvals include the number of approvals in square brackets after the [section name](#section-names). + +```plaintext +[Section requiring 5 approvals][5] +``` + +NOTE: +Optional sections ignore the number of approvals required. + +#### Sections with default owners + +You can define a default owner for the entries in a section by appending the owners to the [section heading](#section-headings). + +```plaintext +# Section with @username as default owner +[Section name] @username + +# Section with @group and @subgroup as default owners and requiring 2 approvals +[Section name][2] @group @subgroup +``` + +### Code Owner entries + +Each Code Owner entry includes a path followed by one or more owners. + +```plaintext +README.md @username1 +``` + +NOTE: +If an entry is duplicated in a section, [the last entry is used from each section.](index.md#define-more-specific-owners-for-more-specifically-defined-files-or-directories) + +### Relative paths + +If a path does not start with a `/`, the path is treated as if it starts with +a [globstar](#globstar-paths). `README.md` is treated the same way as `/**/README.md`: + +```plaintext +# This will match /README.md, /internal/README.md, /app/lib/README.md +README.md @username + +# This will match /internal/README.md, /docs/internal/README.md, /docs/api/internal/README.md +internal/README.md +``` + +### Absolute paths + +If a path starts with a `/` it matches the root of the repository. + +```plaintext +# Matches only the file named `README.md` in the root of the repository. +/README.md + +# Matches only the file named `README.md` inside the `/docs` directory. +/docs/README.md +``` + +### Directory paths + +If a path ends with `/`, the path matches any file in the directory. + +```plaintext +# This is the same as `/docs/**/*` +/docs/ +``` + +### Wildcard paths + +Wildcards can be used to match one of more characters of a path. + +```plaintext +# Any markdown files in the docs directory +/docs/*.md @username + +# /docs/index file of any filetype +# For example: /docs/index.md, /docs/index.html, /docs/index.xml +/docs/index.* @username + +# Any file in the docs directory with 'spec' in the name. +# For example: /docs/qa_specs.rb, /docs/spec_helpers.rb, /docs/runtime.spec +/docs/*spec* @username + +# README.md files one level deep within the docs directory +# For example: /docs/api/README.md +/docs/*/README.md @username +``` + +### Globstar paths + +Globstars (`**`) can be used to match zero or more directories and subdirectories. + +```plaintext +# This will match /docs/index.md, /docs/api/index.md, /docs/api/graphql/index.md +/docs/**/index.md +``` + +### Entry owners + +Entries must be followed by one or more owner. These can be groups, subgroups, +and users. Order of owners is not important. + +```plaintext +/path/to/entry.rb @group +/path/to/entry.rb @group/subgroup +/path/to/entry.rb @user +/path/to/entry.rb @group @group/subgroup @user +``` + +#### Groups as entry owners + +Groups and subgroups can be owners of an entry. +Each entry can be owned by [one or more owners](#entry-owners). +For more details see the [Add a group as a Code Owner](index.md#add-a-group-as-a-code-owner). + +```plaintext +/path/to/entry.rb @group +/path/to/entry.rb @group/subgroup +/path/to/entry.rb @group @group/subgroup +``` + +### Users as entry owners + +Users can be owners of an entry. Each entry can be owned by +[one or more owners](#entry-owners). + +```plaintext +/path/to/entry.rb @username1 +/path/to/entry.rb @username1 @username2 +``` + +## Error handling in Code Owners + +### Entries with spaces + +Paths containing whitespace must be escaped with backslashes: `path\ with\ spaces/*.md`. +Without the backslashes, the path after the first whitespace is parsed as an owner. +GitLab the parses `folder with spaces/*.md @group` into +`path: "folder", owners: " with spaces/*.md @group"`. + +### Unparsable sections + +If a section heading cannot be parsed, the section is: + +1. Parsed as an entry. +1. Added to the previous section. +1. If no previous section exists, the section is added to the default section. + +For example, this file is missing a square closing bracket: + +```plaintext +* @group + +[Section name +docs/ @docs_group +``` + +GitLab recognizes the heading `[Section name` as an entry. The default section includes 3 rules: + +- Default section + - `*` owned by `@group` + - `[Section` owned by `name` + - `docs/` owned by `@docs_group` + +This file contains an unescaped space between the words `Section` and `name`. +GitLab recognizes the intended heading as an entry: + +```plaintext +[Docs] +docs/**/* @group + +[Section name]{2} @group +docs/ @docs_group +``` + +The `[Docs]` section then includes 3 rules: + +- `docs/**/*` owned by `@group` +- `[Section` owned by `name]{2} @group` +- `docs/` owned by `@docs_group` + +### Malformed owners + +Each entry must contain 1 or more owners to be valid, malformed owners are ignored. +For example `/path/* @group user_without_at_symbol @user_with_at_symbol` +is owned by `@group` and `@user_with_at_symbol`. + +### Inaccessible or incorrect owners + +Inaccessible or incorrect owners are ignored. For example, if `@group`, `@username`, +and `example@gitlab.com` are accessible on the project and we create an entry: + +```plaintext +* @group @grou @username @i_left @i_dont_exist example@gitlab.com invalid@gitlab.com +``` + +GitLab ignores `@grou`, `@i_left`, `@i_dont_exist`, and `invalid@gitlab.com`. + +For more information on who is accessible, see [Add a group as a Code Owner](index.md#add-a-group-as-a-code-owner). + +### Zero owners + +If an entry includes no owners, or zero [accessible owners](#inaccessible-or-incorrect-owners) +exist, the entry is invalid. Because this rule can never be satisfied, GitLab +auto-approves it in merge requests. + +NOTE: +When a protected branch has `Require code owner approval` enabled, rules with +zero owners are still honored. + +### Less than 1 required approval + +When [defining the number of approvals](index.md#require-multiple-approvals-from-code-owners) for a section, +the minimum number of approvals is `1`. Setting the number of approvals to +`0` results in GitLab requiring one approval. + +## Example `CODEOWNERS` file + +```plaintext +# This is an example of a CODEOWNERS file. +# Lines that start with `#` are ignored. + +# app/ @commented-rule + +# Specify a default Code Owner by using a wildcard: +* @default-codeowner + +# Specify multiple Code Owners by using a tab or space: +* @multiple @code @owners + +# Rules defined later in the file take precedence over the rules +# defined before. +# For example, for all files with a filename ending in `.rb`: +*.rb @ruby-owner + +# Files with a `#` can still be accessed by escaping the pound sign: +\#file_with_pound.rb @owner-file-with-pound + +# Specify multiple Code Owners separated by spaces or tabs. +# In the following case the CODEOWNERS file from the root of the repo +# has 3 Code Owners (@multiple @code @owners): +CODEOWNERS @multiple @code @owners + +# You can use both usernames or email addresses to match +# users. Everything else is ignored. For example, this code +# specifies the `@legal` and a user with email `janedoe@gitlab.com` as the +# owner for the LICENSE file: +LICENSE @legal this_does_not_match janedoe@gitlab.com + +# Use group names to match groups, and nested groups to specify +# them as owners for a file: +README @group @group/with-nested/subgroup + +# End a path in a `/` to specify the Code Owners for every file +# nested in that directory, on any level: +/docs/ @all-docs + +# End a path in `/*` to specify Code Owners for every file in +# a directory, but not nested deeper. This code matches +# `docs/index.md` but not `docs/projects/index.md`: +/docs/* @root-docs + +# Include `/**` to specify Code Owners for all subdirectories +# in a directory. This rule matches `docs/projects/index.md` or +# `docs/development/index.md` +/docs/**/*.md @root-docs + +# This code makes matches a `lib` directory nested anywhere in the repository: +lib/ @lib-owner + +# This code match only a `config` directory in the root of the repository: +/config/ @config-owner + +# If the path contains spaces, escape them like this: +path\ with\ spaces/ @space-owner + +# Code Owners section: +[Documentation] +ee/docs @docs +docs @docs + +# Use of default owners for a section. In this case, all files (*) are owned by +the dev team except the README.md and data-models which are owned by other teams. +[Development] @dev-team +* +README.md @docs-team +data-models/ @data-science-team + +# This section is combined with the previously defined [Documentation] section: +[DOCUMENTATION] +README.md @docs +``` diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index a68f9550ebf..09a443614d0 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -1,11 +1,11 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: howto, reference --- -# Deploy boards (DEPRECATED) **(FREE)** +# Deploy boards (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1589) in GitLab 9.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index fc88535dc77..13ee07097e1 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -15,8 +15,7 @@ Depending on your needs, you might want to use a [deploy token](../deploy_tokens |------------------|-------------|--------------| | Sharing | Shareable between multiple projects, even those in different groups. | Belong to a project or group. | | Source | Public SSH key generated on an external host. | Generated on your GitLab instance, and is provided to users only at creation time. | -| Validity | Valid as long as it's registered and enabled, and the user that created it exists. | Can be given an expiration date. | -| Registry access | Cannot access a package registry. | Can read from and write to a package registry. | +| Accessible resources | Git repository over SSH | Git repository over HTTP, package registry, and container registry. | Deploy keys can't be used for Git operations if [external authorization](../../admin_area/settings/external_authorization.md) is enabled. @@ -41,10 +40,8 @@ A deploy key is given a permission level when it is created: You can change a deploy key's permission level after creating it. Changing a project deploy key's permissions only applies for the current project. -When a read-write deploy key is used to push a commit, GitLab checks if the creator of the -deploy key has permission to access the resource. - -For example: +Although a deploy key is a secret that isn't associated with a specific user, +GitLab authorizes the creator of the deploy key if the Git-command triggers additional processes. For example: - When a deploy key is used to push a commit to a [protected branch](../protected_branches.md), the _creator_ of the deploy key must have access to the branch. @@ -52,6 +49,15 @@ For example: deploy key must have access to the CI/CD resources, including protected environments and secret variables. +## Security implications + +The intended use case for deploy keys is for non-human interaction with GitLab, for example: an automated script running on a server in your organization. +As with all sensitive information, you should ensure only those who need access to the secret can read it. +For human interactions, use credentials tied to users such as Personal Access Tokens. + +To help detect a potential secret leak, you can use the +[Audit Event](../../../administration/audit_event_streaming.md#example-payloads-for-ssh-events-with-deploy-key) feature. + ## View deploy keys To view the deploy keys available to a project: @@ -80,6 +86,7 @@ Prerequisites: 1. Complete the fields. 1. Optional. To grant `read-write` permission, select the **Grant write permissions to this key** checkbox. +1. Optional. Update the **Expiration date**. A project deploy key is enabled when it is created. You can modify only a project deploy key's name and permissions. @@ -88,9 +95,9 @@ name and permissions. Prerequisites: -- You must have administrator access. -- [Generate an SSH key pair](../../ssh.md#generate-an-ssh-key-pair). Put the private SSH - key on the host that requires access to the repository. +- You must have administrator access to the instance. +- You must [generate an SSH key pair](../../ssh.md#generate-an-ssh-key-pair). +- You must put the private SSH key on the host that requires access to the repository. To create a public deploy key: @@ -153,7 +160,7 @@ There are a few scenarios where a deploy key fails to push to a - The owner associated to a deploy key does not have access to the protected branch. - The owner associated to a deploy key does not have [membership](../members/index.md) to the project of the protected branch. -- **No one** is selected in [the **Allowed to push** section](../protected_branches.md#configure-a-protected-branch) of the protected branch. +- **No one** is selected in [the **Allowed to push and merge** section](../protected_branches.md#configure-a-protected-branch) of the protected branch. All deploy keys are associated to an account. Since the permissions for an account can change, this might lead to scenarios where a deploy key that was working is suddenly unable to push to a protected branch. diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index cfb382d73e2..a81ccd043bd 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index ffbe7447aa8..f0ced95c8eb 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -16,7 +16,7 @@ You might want to use these templates: - For different stages of your workflow, for example, feature proposal, feature improvement, or a bug report. - For every issue or merge request for a specific project, so the layout is consistent. -- For a [Service Desk email template](service_desk.md#new-service-desk-issues). +- For a [Service Desk email template](service_desk.md#use-a-custom-template-for-service-desk-issues). For description templates to work, they must be: @@ -46,10 +46,11 @@ and see if you can find your description template in the **Choose a template** d ## Create a merge request template Similarly to issue templates, create a new Markdown (`.md`) file inside the -`.gitlab/merge_request_templates/` directory in your repository. Commit and -push to your default branch. +`.gitlab/merge_request_templates/` directory in your repository. Unlike issue +templates, merge requests have [additional inheritance rules](merge_requests/creating_merge_requests.md) +that depend on the contents of commit messages and branch names. -To create a merge request description template: +To create a merge request description template for a project: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository**. @@ -87,6 +88,10 @@ For example: `https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_templat > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89810) in GitLab 15.7. +NOTE: +This feature is available only for +[the default template](#set-a-default-template-for-merge-requests-and-issues). + When you save a merge request for the first time, GitLab replaces these variables in your merge request template with their values: @@ -95,7 +100,7 @@ your merge request template with their values: | `%{all_commits}` | Messages from all commits in the merge request. Limited to 100 most recent commits. Skips commit bodies exceeding 100 KiB and merge commit messages. | `* Feature introduced` <br><br> `This commit implements feature` <br> `Changelog:added` <br><br> `* Bug fixed` <br><br> `* Documentation improved` <br><br>`This commit introduced better docs.` | | `%{co_authored_by}` | Names and emails of commit authors in a `Co-authored-by` Git commit trailer format. Limited to authors of 100 most recent commits in merge request. | `Co-authored-by: Zane Doe <zdoe@example.com>` <br> `Co-authored-by: Blake Smith <bsmith@example.com>` | | `%{first_commit}` | Full message of the first commit in merge request diff. | `Update README.md` | -| `%{first_multiline_commit}` | Full message of the first commit that's not a merge commit and has more than one line in message body. Merge request title if all commits aren't multiline. | `Update README.md`<br><br>`Improved project description in readme file.` | +| `%{first_multiline_commit}` | Full message of the first commit that's not a merge commit and has more than one line in message body. Merge request title if all commits aren't multiline. | `Update README.md` <br><br> `Improved project description in readme file.` | | `%{source_branch}` | The name of the branch being merged. | `my-feature-branch` | | `%{target_branch}` | The name of the branch that the changes are applied to. | `main` | @@ -148,11 +153,11 @@ To set a default description template for merge requests, either: - [In GitLab 14.8 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78302), [create a merge request template](#create-a-merge-request-template) named `Default.md` (case insensitive) and save it in `.gitlab/merge_request_templates/`. This [doesn't overwrite](#priority-of-default-description-templates) the default template if one has been set in the project settings. -- Users on GitLab Premium and higher: set the default template in project settings: +- Users on GitLab Premium and Ultimate: set the default template in project settings: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Merge requests**. - 1. In the **Merge commit message template** section, fill in **Default description template for merge requests**. + 1. In the **Default description template for merge requests** section, fill in the text area. 1. Select **Save changes**. To set a default description template for issues, either: @@ -160,12 +165,12 @@ To set a default description template for issues, either: - [In GitLab 14.8 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78302), [create an issue template](#create-an-issue-template) named `Default.md` (case insensitive) and save it in `.gitlab/issue_templates/`. This [doesn't overwrite](#priority-of-default-description-templates) the default template if one has been set in the project settings. -- Users on GitLab Premium and higher: set the default template in project settings: +- Users on GitLab Premium and Ultimate: set the default template in project settings: 1. On the top bar, select **Main menu > Projects** and find your project. - 1. On the left sidebar, select **Settings**. - 1. Expand **Default issue template**. - 1. Fill in the **Default description template for issues** text area. + 1. On the left sidebar, select **Settings > General**. + 1. Expand **Default description template for issues**. + 1. Fill in the text area. 1. Select **Save changes**. Because GitLab merge request and issues support [Markdown](../markdown.md), you can use it to format @@ -176,7 +181,7 @@ You can also provide `issues_template` and `merge_requests_template` attributes #### Priority of default description templates -When you set [merge request and issue description templates](#set-a-default-template-for-merge-requests-and-issues) +When you set [issue description templates](#set-a-default-template-for-merge-requests-and-issues) in various places, they have the following priorities in a project. The ones higher up override the ones below: @@ -184,6 +189,9 @@ The ones higher up override the ones below: 1. `Default.md` (case insensitive) from the parent group. 1. `Default.md` (case insensitive) from the project repository. +Merge requests have [additional inheritance rules](merge_requests/creating_merge_requests.md) +that depend on the contents of commit messages and branch names. + ## Example description template We use description templates for issues and merge requests in the diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 8d3446994e8..c81ba4b7dd3 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -58,7 +58,7 @@ git-lfs --version ``` If it doesn't recognize this command, you must install it. There are -several [installation methods](https://git-lfs.github.com/) that you can +several [installation methods](https://git-lfs.com/) that you can choose according to your OS. To install it with Homebrew: ```shell @@ -195,8 +195,7 @@ Suggested workflow for shared projects: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in GitLab 8.9. This process allows you to lock one file at a time through the GitLab UI and -requires access to [GitLab Premium](https://about.gitlab.com/pricing/) -or higher tiers. +requires access to the [GitLab Premium or Ultimate tier](https://about.gitlab.com/pricing/). Default branch file and directory locks only apply to the [default branch](repository/branches/default.md) set in the project's settings. @@ -209,7 +208,7 @@ requests that modify locked files. Unlock the file to allow changes. To lock a file: 1. Open the file or directory in GitLab. -1. In the upper right, above the file, select **Lock**. +1. In the upper-right corner, above the file, select **Lock**. 1. On the confirmation dialog box, select **OK**. If you do not have permission to lock the file, the button is not enabled. @@ -221,7 +220,7 @@ To view the user who locked the file (if it was not you), hover over the button. To view and remove file locks: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Repository > Locked Files**. +1. On the left sidebar, select **Repository > Locked files**. This list shows all the files locked either through LFS or GitLab UI. diff --git a/doc/user/project/git_attributes.md b/doc/user/project/git_attributes.md index 1feb17b19c8..698b888a32a 100644 --- a/doc/user/project/git_attributes.md +++ b/doc/user/project/git_attributes.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Git Attributes **(FREE)** +# Git attributes **(FREE)** GitLab supports defining custom [Git attributes](https://git-scm.com/docs/gitattributes) such as what files to treat as binary, and what language to use for syntax highlighting @@ -14,14 +14,41 @@ diffs. To define these attributes, create a file called `.gitattributes` in the root directory of your repository and push it to the default branch of your project. -## Encoding Requirements +## Encoding requirements The `.gitattributes` file _must_ be encoded in UTF-8 and _must not_ contain a Byte Order Mark. If a different encoding is used, the file's contents are ignored. -## Syntax Highlighting +## Support for mixed file encodings + +GitLab attempts to detect the encoding of files automatically, but defaults to UTF-8 unless +the detector is confident of a different type (such as `ISO-8859-1`). Incorrect encoding +detection can result in some characters not displaying in the text, such as accented characters in a +non-UTF-8 encoding. + +Git has built-in support for handling this eventuality and automatically converts files between +a designated encoding and UTF-8 for the repository itself. Configure support for mixed file encoding in the `.gitattributes` +file using the `working-tree-encoding` attribute. + +Example: + +```plaintext +*.xhtml text working-tree-encoding=ISO-8859-1 +``` + +With this example configuration, Git maintains all `.xhtml` files in the repository in ISO-8859-1 +encoding in the local tree, but converts to and from UTF-8 when committing into the repository. GitLab +renders the files accurately as it only sees correctly encoded UTF-8. + +If applying this configuration to an existing repository, files may need to be touched and recommitted +if the local copy has the correct encoding but the repository does not. This can +be performed for the whole repository by running `git add --renormalize .`. + +For more information, see [working-tree-encoding](https://git-scm.com/docs/gitattributes#_working_tree_encoding). + +## Syntax highlighting The `.gitattributes` file can be used to define which language to use when -syntax highlighting files and diffs. See -["Syntax Highlighting"](highlighting.md) for more information. +syntax highlighting files and diffs. For more information, see +[Syntax highlighting](highlighting.md). diff --git a/doc/user/project/img/codeowners_in_UI_v15_10.png b/doc/user/project/img/codeowners_in_UI_v15_10.png Binary files differnew file mode 100644 index 00000000000..c643d333791 --- /dev/null +++ b/doc/user/project/img/codeowners_in_UI_v15_10.png diff --git a/doc/user/project/img/issue_board_system_notes_v13_6.png b/doc/user/project/img/issue_board_system_notes_v13_6.png Binary files differdeleted file mode 100644 index 4958f63541e..00000000000 --- a/doc/user/project/img/issue_board_system_notes_v13_6.png +++ /dev/null diff --git a/doc/user/project/img/multi_approvals_code_owners_sections_v15_9.png b/doc/user/project/img/multi_approvals_code_owners_sections_v15_9.png Binary files differnew file mode 100644 index 00000000000..a7fea76d5b1 --- /dev/null +++ b/doc/user/project/img/multi_approvals_code_owners_sections_v15_9.png diff --git a/doc/user/project/img/protected_tags_list_v12_3.png b/doc/user/project/img/protected_tags_list_v12_3.png Binary files differdeleted file mode 100644 index 6a30f615f2f..00000000000 --- a/doc/user/project/img/protected_tags_list_v12_3.png +++ /dev/null diff --git a/doc/user/project/img/protected_tags_page_v12_3.png b/doc/user/project/img/protected_tags_page_v12_3.png Binary files differdeleted file mode 100644 index 841e19af8a7..00000000000 --- a/doc/user/project/img/protected_tags_page_v12_3.png +++ /dev/null diff --git a/doc/user/project/img/protected_tags_permissions_dropdown_v12_3.png b/doc/user/project/img/protected_tags_permissions_dropdown_v12_3.png Binary files differdeleted file mode 100644 index 913d4725d53..00000000000 --- a/doc/user/project/img/protected_tags_permissions_dropdown_v12_3.png +++ /dev/null diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index 7114974d8db..b523e007f0e 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -1,7 +1,6 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -33,10 +32,13 @@ When importing: ## Prerequisites +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + - [Bitbucket Cloud integration](../../../integration/bitbucket.md) must be enabled. If that integration is not enabled, ask your GitLab administrator to enable it. The Bitbucket Cloud integration is enabled by default on GitLab.com. -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +- [Bitbucket Cloud import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your + GitLab administrator to enable it. The Bitbucket Cloud import source is enabled by default on GitLab.com. +- At least the Maintainer role on the destination group to import to. ## How it works diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index e6d2e3e00b6..4d3a6eb87e0 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -1,7 +1,6 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -26,11 +25,18 @@ created as private in GitLab as well. > Ability to re-import projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23905) in GitLab 15.9. -Prerequisites: +You can import Bitbucket repositories to GitLab. -- An administrator must enable **Bitbucket Server** in **Admin > Settings > General > Visibility and access controls > Import sources**. -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +### Prerequisites + +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + +- [Bitbucket Server import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The Bitbucket Server import source is enabled + by default on GitLab.com. +- At least the Maintainer role on the destination group to import to. + +### Import repositories To import your Bitbucket repositories: @@ -120,7 +126,16 @@ If the project import completes but LFS objects can't be downloaded or cloned, y password or personal access token containing special characters. For more information, see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337769). +### Pull requests are missing + +Importing large projects spawns a process that can consume a lot of memory. Sometimes you can see messages such as `Sidekiq worker RSS out of range` in the +[Sidekiq logs](../../../administration/logs/index.md#sidekiq-logs). This can mean that MemoryKiller is interrupting the `RepositoryImportWorker` because it's using +too much memory. + +To resolve this problem, temporarily increase the `SIDEKIQ_MEMORY_KILLER_MAX_RSS` environment variable using +[custom environment variables](https://docs.gitlab.com/omnibus/settings/environment-variables.html) from the default `2000000` value to a larger value like `3000000`. +Consider memory constraints on the system before deciding to increase `SIDEKIQ_MEMORY_KILLER_MAX_RSS`. + ## Related topics -For information on automating user, group, and project import API calls, see -[Automate group and project import](index.md#automate-group-and-project-import). +- [Automate group and project import](index.md#automate-group-and-project-import) diff --git a/doc/user/project/import/clearcase.md b/doc/user/project/import/clearcase.md index 0878476d59f..35ed7a043d0 100644 --- a/doc/user/project/import/clearcase.md +++ b/doc/user/project/import/clearcase.md @@ -1,7 +1,6 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index 00aebb75a50..fb3416a3492 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -1,14 +1,13 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Migrating from CVS **(FREE)** [CVS](https://savannah.nongnu.org/projects/cvs) is an old centralized version -control system similar to [SVN](svn.md). +control system similar to [SVN](https://subversion.apache.org/). ## CVS vs Git @@ -71,6 +70,5 @@ Here's a few links to get you started with the migration: - [Migrate using the `cvs-fast-export` tool](https://gitlab.com/esr/cvs-fast-export) - [Stack Overflow post on importing the CVS repository](https://stackoverflow.com/a/11490134/974710) -- [Convert a CVS repository to Git](https://www.techrepublic.com/article/convert-cvs-repositories-to-git/) - [Man page of the `git-cvsimport` tool](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-cvsimport.html) - [Migrate using `reposurgeon`](http://www.catb.org/~esr/reposurgeon/repository-editing.html#conversion) diff --git a/doc/user/project/import/fogbugz.md b/doc/user/project/import/fogbugz.md index d9e03662434..692ec1390d2 100644 --- a/doc/user/project/import/fogbugz.md +++ b/doc/user/project/import/fogbugz.md @@ -1,7 +1,6 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -16,10 +15,16 @@ The importer imports all of your cases and comments with the original case numbers and timestamps. You can also map FogBugz users to GitLab users. -Prerequisite: +## Prerequisites -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + +- [FogBugz import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The FogBugz import source is enabled + by default on GitLab.com. +- At least the Maintainer role on the destination group to import to. + +## Import project from FogBugz To import your project from FogBugz: diff --git a/doc/user/project/import/gitea.md b/doc/user/project/import/gitea.md index 404bb4c8600..62cda62c2fe 100644 --- a/doc/user/project/import/gitea.md +++ b/doc/user/project/import/gitea.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -8,15 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381902) in GitLab 15.8, GitLab no longer automatically creates namespaces or groups that don't exist. GitLab also no longer falls back to using the user's personal namespace if the namespace or group name is taken. -Import your projects from Gitea to GitLab with minimal effort. - -NOTE: -This requires Gitea `v1.0.0` or later. - -Prerequisite: - -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +Import your projects from Gitea to GitLab. The Gitea importer can import: @@ -30,13 +22,21 @@ The Gitea importer can import: When importing, repository public access is retained. If a repository is private in Gitea, it's created as private in GitLab as well. -## How it works - Because Gitea isn't an OAuth provider, author/assignee can't be mapped to users in your GitLab instance. This means the project creator (usually the user that started the import process) is set as the author. A reference, however, is kept on the issue about the original Gitea author. +## Prerequisites + +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + +- Gitea version 1.0.0 or later. +- [Gitea import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The Gitea import source is enabled + by default on GitLab.com. +- At least the Maintainer role on the destination group to import to. + ## Import your Gitea repositories The importer page is visible when you create a new project. @@ -75,10 +75,9 @@ From there, you can view the import statuses of your Gitea repositories: You also can: -- Import all of your Gitea projects in one go by selecting **Import all projects** - in the upper left corner. -- Filter projects by name. If filter is applied, selecting **Import all projects** - imports only matched projects. +- In the upper-left corner, select **Import all projects** to import all of your Gitea projects at once. +- Filter projects by name. If a filter is applied, **Import all projects** + imports only selected projects. ![Gitea importer page](img/import_projects_from_gitea_importer_v12_3.png) diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index eeebb5a166c..b2b1ede12d4 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -1,13 +1,14 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Import your project from GitHub to GitLab **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381902) in GitLab 15.8, GitLab no longer automatically creates namespaces or groups that don't exist. GitLab also no longer falls back to using the user's personal namespace if the namespace or group name is taken. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381902) in GitLab 15.8, GitLab no longer automatically creates namespaces or groups that don't exist. GitLab also no longer falls back to using the user's personal namespace if the namespace or group name is taken. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388716) in GitLab 15.10, you no longer need to add any users to the parent group in GitLab to successfully import the **Require a pull request before merging - Allow specified actors to bypass required pull requests** branch protection rule. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378267) in GitLab 15.11, GitLab instances behind proxies no longer require `github.com` and `api.github.com` entries in the [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains). You can import your GitHub projects from either GitHub.com or GitHub Enterprise. Importing projects does not migrate or import any types of groups or organizations from GitHub to GitLab. @@ -30,18 +31,19 @@ When importing projects: imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to a discrepancy in branches compared to those of the GitHub repository. -For additional technical details, refer to the [GitHub Importer](../../../development/github_importer.md) -developer documentation. - -For an overview of the import process, see the video [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview of the import process, see [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). ## Prerequisites +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + To import projects from GitHub: -- You must have at least the Maintainer role on the destination group to import to. Using the Developer role for this - purpose was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in - GitLab 16.0. +- [GitHub import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The GitHub import source is enabled + by default on GitLab.com. +- You must have at least the Maintainer role on the destination group to import to. - Each GitHub author and assignee in the repository must have a [public-facing email address](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address) on GitHub that matches their GitLab email address (regardless of how the account was created). If their email address @@ -54,17 +56,18 @@ To import projects from GitHub: are properly mapped to the same user in GitLab. GitHub Enterprise does not require this field to be populated so you may have to add it on existing accounts. -See also [Branch protection rules and project settings](#branch-protection-rules-and-project-settings) for additional -prerequisites for those imports. +Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383047), if you are using GitHub as an OmniAuth provider, ensure that the URL +perimeter is specified in the [OmniAuth configuration](../../../integration/github.md#enable-github-oauth-in-gitlab). ### Importing from GitHub Enterprise to self-managed GitLab If you are importing from GitHub Enterprise to a self-managed GitLab instance: - You must first enable the [GitHub integration](../../../integration/github.md). -- If GitLab is behind a HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#create-an-allowlist-for-local-requests) - with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue - [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941). +- GitHub must be enabled as an import source in the + [Admin Area](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). +- For GitLab 15.10 and earlier, you must add `github.com` and `api.github.com` entries in the + [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains). ### Importing from GitHub.com to self-managed GitLab @@ -82,10 +85,9 @@ Before you begin, ensure that any GitHub user you want to map to a GitLab user h [publicly visible email address](https://docs.github.com/en/rest/users#get-a-user) on GitHub. If you are importing to GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#use-a-github-token). -We do not recommend this method, as it does not associate all user activity (such as issues and pull requests) with matching GitLab users. -User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with -the user account that is performing the import. +If a GitHub user's public email address doesn't match any GitLab user email address, the user's activity is associated with the user account that is +performing the import. NOTE: If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured @@ -103,9 +105,7 @@ Prerequisite: - Authentication token with administrator access. -Using a personal access token to import projects is not recommended. If you are a GitLab.com user, -you can use a personal access token to import your project from GitHub, but this method cannot -associate all user activity (such as issues and pull requests) with matching GitLab users. +If you are a GitLab.com user, you can use a personal access token to import your project from GitHub. If you are an administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, you cannot use a personal access token. The [GitHub integration method (above)](#use-the-github-integration) is recommended for all users. @@ -124,6 +124,21 @@ If you are not using the GitHub integration, you can still perform an authorizat To use a newer personal access token in imports after previously performing these steps, sign out of your GitLab account and sign in again, or revoke the older personal access token in GitHub. +### Filter repositories list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385113) in GitLab 16.0. + +After you authorize access to your GitHub repositories, GitLab redirects you to the importer page and +your GitHub repositories are listed. + +Use one of the following tabs to filter the list of repositories: + +- **Owner** (default): Filter the list to the repositories that you are the owner of. +- **Collaborated**: Filter the list to the repositories that you have contributed to. +- **Organization**: Filter the list to the repositories that belong to an organization you are a member of. + +When the **Organization** tab is selected, you can further narrow down your search by selecting an available GitHub organization from a dropdown. + ### Select additional items to import > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/373705) in GitLab 15.5. @@ -140,15 +155,15 @@ You can choose to import these items, but this could significantly increase impo - **Import issue and pull request events**. - **Use alternative comments import method**. - **Import Markdown attachments**. +- **Import collaborators** (selected by default). Leaving it selected might result in new users using a seat in the group or namespace, + and being granted permissions [as high as project owner](#collaborators-members). Only direct collaborators are imported. + Outside collaborators are never imported. ### Select which repositories to import > - Ability to cancel pending or active imports [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247325) in GitLab 15.7. > - Ability to re-import projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23905) in GitLab 15.9. -After you have authorized access to your GitHub repositories, you are redirected to the GitHub importer page and -your GitHub repositories are listed. - By default, the proposed repository namespaces match the names as they exist in GitHub, but based on your permissions, you can choose to edit these names before you proceed to import any of them. @@ -170,6 +185,18 @@ Completed imports can be re-imported by selecting **Re-import** and specifying n ![GitHub importer page](img/import_projects_from_github_importer_v12_3.png) +### Check status of imports + +> Details of partially completed imports with a list of entities that failed to import [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386748) in GitLab 16.0. + +After imports are completed, they can be in one of three states: + +- **Completed**: GitLab imported all repository entities. +- **Partially completed**: GitLab failed to import some repository entities. +- **Failed**: GitLab imported no repository entities. + +Expand **Details** to see a list of [repository entities](#imported-data) that failed to import. + ## Mirror a repository and share pipeline status **(PREMIUM)** Depending on your GitLab tier, [repository mirroring](../repository/mirror/index.md) can be set up to keep @@ -209,17 +236,19 @@ The following items of a project are imported: - Repository description. - Git repository data. - Branch protection rules. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22650) in GitLab 15.4. +- Collaborators (members). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388716) in GitLab 15.10. From GitLab 16.0, can + be imported [as an additional item](#select-additional-items-to-import). - Issues. - Pull requests. - Wiki pages. - Milestones. - Labels. -- Release note descriptions. +- Release notes content. - Attachments for: - Release notes. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15620) in GitLab 15.4. - - Comments and notes. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5. + - Comments. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5. - Issue description. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5. - - Merge Request description. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5. + - Pull Request description. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5. All attachment imports are disabled by default behind `github_importer_attachments_import` [feature flag](../../../administration/feature_flags.md). From GitLab 15.5, can @@ -232,7 +261,7 @@ The following items of a project are imported: - Pull request "merged by" information. - Pull request comments replies in discussions. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336596) in GitLab 14.5. -- Diff Notes suggestions. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340624) in GitLab 14.7. +- Pull request review comments suggestions. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340624) in GitLab 14.7. - Issue events and pull requests events. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7673) in GitLab 15.4 with `github_importer_issue_events_import` [feature flag](../../../administration/feature_flags.md) disabled by default. From GitLab 15.5, can be imported [as an additional item](#select-additional-items-to-import). The feature flag was @@ -252,14 +281,11 @@ When they are imported, supported GitHub branch protection rules are mapped to e | GitHub rule | GitLab rule | Introduced in | | :---------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------ | | **Require conversation resolution before merging** for the project's default branch | **All threads must be resolved** [project setting](../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved) | [GitLab 15.5](https://gitlab.com/gitlab-org/gitlab/-/issues/371110) | -| **Require a pull request before merging** | **No one** option in the **Allowed to push** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) | [GitLab 15.5](https://gitlab.com/gitlab-org/gitlab/-/issues/370951) | +| **Require a pull request before merging** | **No one** option in the **Allowed to push and merge** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) | [GitLab 15.5](https://gitlab.com/gitlab-org/gitlab/-/issues/370951) | | **Require signed commits** for the project's default branch | **Reject unsigned commits** GitLab [push rule](../repository/push_rules.md#prevent-unintended-consequences) **(PREMIUM)** | [GitLab 15.5](https://gitlab.com/gitlab-org/gitlab/-/issues/370949) | | **Allow force pushes - Everyone** | **Allowed to force push** [branch protection setting](../protected_branches.md#allow-force-push-on-a-protected-branch) | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/370943) | | **Require a pull request before merging - Require review from Code Owners** | **Require approval from code owners** [branch protection setting](../protected_branches.md#require-code-owner-approval-on-a-protected-branch) **(PREMIUM)** | [GitLab 15.6](https://gitlab.com/gitlab-org/gitlab/-/issues/376683) | -| **Require a pull request before merging - Allow specified actors to bypass required pull requests** (1) | List of users in the **Allowed to push** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) **(PREMIUM)**. Without a Premium license, the list of users that are allowed to push is limited to roles. | [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/384939) | - -1. To successfully import the **Require a pull request before merging - Allow specified actors to bypass required pull requests** rule, you must add to the parent group in GitLab - the users that are allowed to bypass required pull requests before you begin importing. +| **Require a pull request before merging - Allow specified actors to bypass required pull requests** | List of users in the **Allowed to push and merge** list of [branch protection settings](../protected_branches.md#configure-a-protected-branch) **(PREMIUM)**. Without a **Premium** subscription, the list of users that are allowed to push and merge is limited to roles. | [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/384939) | Mapping GitHub rule **Require status checks to pass before merging** to [external status checks](../merge_requests/status_checks.md) was considered in issue @@ -267,39 +293,25 @@ Mapping GitHub rule **Require status checks to pass before merging** to into GitLab due to technical difficulties. You can still create [external status checks](../merge_requests/status_checks.md) manually. -## Alternative way to import notes and diff notes - -When GitHub Importer runs on extremely large projects not all notes & diff notes can be imported due to GitHub API `issues_comments` & `pull_requests_comments` endpoints limitation. -Not all pages can be fetched due to the following error coming from GitHub API: `In order to keep the API fast for everyone, pagination is limited for this resource. Check the rel=last link relation in the Link response header to see how far back you can traverse.`. - -An [alternative approach](#select-additional-items-to-import) for importing comments is available. - -Instead of using `issues_comments` and `pull_requests_comments`, use individual resources `issue_comments` and `pull_request_comments` instead to pull notes from one object at a time. -This allows us to carry over any missing comments, however it increases the number of network requests required to perform the import, which means its execution takes a longer time. +### Collaborators (members) -## Reduce GitHub API request objects per page +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388716) in GitLab 15.10. -Some GitHub API endpoints may return a 500 or 502 error for project imports from large repositories. -To reduce the chance of such errors, you can enable the feature flag -`github_importer_lower_per_page_limit` in the group project importing the data. This reduces the -page size from 100 to 50. +These GitHub collaborator roles are mapped to these GitLab [member roles](../../permissions.md#roles): -To enable this feature flag, start a [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session) -and run the following `enable` command: +| GitHub role | Mapped GitLab role | +|:------------|:-------------------| +| Read | Guest | +| Triage | Reporter | +| Write | Developer | +| Maintain | Maintainer | +| Admin | Owner | -```ruby -group = Group.find_by_full_path('my/group/fullpath') +GitHub Enterprise Cloud has +[custom repository roles](https://docs.github.com/en/enterprise-cloud@latest/organizations/managing-peoples-access-to-your-organization-with-roles/about-custom-repository-roles). +These roles aren't supported and cause partially completed imports. -# Enable -Feature.enable(:github_importer_lower_per_page_limit, group) -``` - -To disable the feature, run this command: - -```ruby -# Disable -Feature.disable(:github_importer_lower_per_page_limit, group) -``` +To import GitHub collaborators, you must have at least the Write role on the GitHub project. Otherwise collaborators import is skipped. ## Import from GitHub Enterprise on an internal network @@ -443,3 +455,47 @@ repository to be imported manually. Administrators can manually import the repos # Trigger import from second step Gitlab::GithubImport::Stage::ImportRepositoryWorker.perform_async(project.id) ``` + +### Errors when importing large projects + +The GitHub importer might encounter some errors when importing large projects. + +#### Alternative way to import notes and diff notes + +When the GitHub importer runs on extremely large projects, not all notes and diff notes can be imported due to the GitHub API `issues_comments` and `pull_requests_comments` endpoint limitations. + +If it's not possible to fetch all pages, the GitHub API might return the following error: + +```plaintext +In order to keep the API fast for everyone, pagination is limited for this resource. Check the rel=last link relation in the Link response header to see how far back you can traverse. +``` + +An [alternative approach](#select-additional-items-to-import) for importing comments is available. + +Instead of using `issues_comments` and `pull_requests_comments`, use individual resources to pull notes from one object at a time. This way, you can carry over any missing comments. However, execution takes longer because this method increases the number of network requests required to perform the import. + +#### Reduce GitHub API request objects per page + +Some GitHub API endpoints might return a `500` or `502` error for project imports from large repositories. +To reduce the chance of these errors, in the group project importing the data, enable the +`github_importer_lower_per_page_limit` feature flag. When enabled, the flag reduces the +page size from `100` to `50`. + +To enable this feature flag: + +1. Start a [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session). +1. Run the following `enable` command: + + ```ruby + group = Group.find_by_full_path('my/group/fullpath') + + # Enable + Feature.enable(:github_importer_lower_per_page_limit, group) + ``` + +To disable the feature flag, run this command: + +```ruby +# Disable +Feature.disable(:github_importer_lower_per_page_limit, group) +``` diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index c00709d9697..2b69cd40fc7 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -1,39 +1,17 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Import a project from GitLab.com to your self-managed GitLab instance (deprecated) **(FREE)** +# Import a project from GitLab.com to your self-managed GitLab instance (removed) **(FREE)** WARNING: The GitLab.com importer was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108502) in GitLab 15.8 -and will be removed in GitLab 16.0. To import GitLab projects from GitLab.com to a self-managed GitLab instance use +and removed in GitLab 16.0. To import GitLab projects from GitLab.com to a self-managed GitLab instance use [migrating groups and projects by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). -You can import your existing GitLab.com projects to your GitLab instance. - -Prerequisite: - -- GitLab.com integration must be enabled on your GitLab instance. - [Read more about GitLab.com integration for self-managed GitLab instances](../../../integration/gitlab.md). - -To import a GitLab.com project to your self-managed GitLab instance: - -1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. -1. On the right of the page, select **New project**. -1. Select **Import project**. -1. Select **GitLab.com**. -1. Give GitLab.com permission to access your projects. -1. Select **Import**. - -The importer imports your repository and issues. -When the importer is done, a new GitLab project is created with your imported data. - ## Related topics -- To automate user, group, and project import API calls, see - [Automate group and project import](index.md#automate-group-and-project-import). -- To import Wiki and merge request data to your new instance, - see [exporting a project](../settings/import_export.md#export-a-project-and-its-data). +- [Automate group and project import](index.md#automate-group-and-project-import) +- [Export a project](../settings/import_export.md#export-a-project-and-its-data) diff --git a/doc/user/project/import/img/bitbucket_import_select_project_v12_3.png b/doc/user/project/import/img/bitbucket_import_select_project_v12_3.png Binary files differdeleted file mode 100644 index bbc72a0b4b7..00000000000 --- a/doc/user/project/import/img/bitbucket_import_select_project_v12_3.png +++ /dev/null diff --git a/doc/user/project/import/img/fogbugz_import_finished.png b/doc/user/project/import/img/fogbugz_import_finished.png Binary files differdeleted file mode 100644 index 62c5c86c9b3..00000000000 --- a/doc/user/project/import/img/fogbugz_import_finished.png +++ /dev/null diff --git a/doc/user/project/import/img/manifest_status_v13_3.png b/doc/user/project/import/img/manifest_status_v13_3.png Binary files differdeleted file mode 100644 index c1a55ba1f50..00000000000 --- a/doc/user/project/import/img/manifest_status_v13_3.png +++ /dev/null diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 24748b2e89c..1265b8534d0 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -1,7 +1,6 @@ --- -type: reference, howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -15,16 +14,11 @@ If you want to bring existing projects to GitLab or copy GitLab projects to a di - Between a self-managed instance and GitLab.com in both directions. - In the same GitLab instance. -Prerequisite: - -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. - For any type of source and target, you can migrate GitLab projects: - When [migrating groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended), which allows you to migrate all projects in a group simultaneously. Migrating projects by direct transfer is in - [Beta](../../../policy/alpha-beta-support.md#beta-features). The feature is not ready for production use. + [Beta](../../../policy/alpha-beta-support.md#beta). The feature is not ready for production use. - Using [file exports](../settings/import_export.md). With this method you can migrate projects one by one. No network connection between instances is required. @@ -32,13 +26,27 @@ If you only need to migrate Git repositories, you can [import each project by UR import issues and merge requests this way. To retain metadata like issues and merge requests, either: - [Migrate projects with groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). - This feature is in [Beta](../../../policy/alpha-beta-support.md#beta-features). It is not ready for production use. + This feature is in [Beta](../../../policy/alpha-beta-support.md#beta). It is not ready for production use. - Use [file exports](../settings/import_export.md) to import projects. Keep in mind the limitations of [migrating using file exports](../settings/import_export.md#items-that-are-exported). When migrating from self-managed to GitLab.com, user associations (such as comment author) are changed to the user who is importing the projects. +## Security + +Only import projects from sources you trust. If you import a project from an untrusted source, +an attacker could steal your sensitive data. For example, an imported project +with a malicious `.gitlab-ci.yml` file could allow an attacker to exfiltrate group CI/CD variables. + +GitLab self-managed administrators can reduce their attack surface by disabling import sources they don't need: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility and access controls**. +1. Scroll to **Import sources**. +1. Clear checkboxes for importers that are not required. + ## Available project importers You can import projects from: @@ -65,7 +73,7 @@ You can then [connect your external repository to get CI/CD benefits](../../../c GitLab can not automatically migrate Subversion repositories to Git. Converting Subversion repositories to Git can be difficult, but several tools exist including: -- [`git svn`](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git), for very small and simple repositories. +- [`git svn`](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git), for very small and basic repositories. - [`reposurgeon`](http://www.catb.org/~esr/reposurgeon/repository-editing.html), for larger and more complex repositories. ## Migrate using the API @@ -82,10 +90,9 @@ over a series of Docker pulls and pushes. Re-run any CI pipelines to retrieve an ## Migrate between two self-managed GitLab instances -To migrate from an existing self-managed GitLab instance to a new self-managed GitLab instance, it's -best to [back up](../../../raketasks/backup_restore.md) -the existing instance and restore it on the new instance. For example, this is useful when migrating -a self-managed instance from an old server to a new server. +To migrate from an existing self-managed GitLab instance to a new self-managed GitLab instance, +you should [back up](../../../raketasks/backup_restore.md) +the existing instance and restore it on the new instance. For example, you could use this method to migrate a self-managed instance from an old server to a new server. The backups produced don't depend on the operating system running GitLab. You can therefore use the restore method to switch between different operating system distributions or versions, as long @@ -109,7 +116,7 @@ To view project import history: 1. On the top bar, select **Create new...** (**{plus-square}**). 1. Select **New project/repository**. 1. Select **Import project**. -1. Select **History** in the upper right corner. +1. In the upper-right corner, select **History**. 1. If there are any errors for a particular import, you can see them by selecting **Details**. The history also includes projects created from [built-in](../index.md#create-a-project-from-a-built-in-template) @@ -154,8 +161,20 @@ GitLab from: For more information, see: +- Information on paid GitLab [migration services](https://about.gitlab.com/services/migration/). - [Quick Start](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/docs/using-congregate.md#quick-start). - [Frequently Asked Migration Questions](https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/blob/master/customer/famq.md), including settings that need checking afterwards and other limitations. For support, customers must enter into a paid engagement with GitLab Professional Services. + +## Troubleshooting + +### Imported repository is missing branches + +If an imported repository does not contain all branches of the source repository: + +1. Set the [environment variable](../../../administration/logs/index.md#override-default-log-level) `IMPORT_DEBUG=true`. +1. Retry the import with a [different group, subgroup, or project name](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#re-import-projects-from-external-providers). +1. If some branches are still missing, inspect [`importer.log`](../../../administration/logs/index.md#importerlog) + (for example, with [`jq`](../../../administration/logs/log_parsing.md#parsing-gitlab-railsimporterlog)). diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index c8b717d0421..ede9eb244c6 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -37,16 +37,10 @@ iterations of the GitLab Jira importer. ## Prerequisites -### Permissions - -To be able to import issues from a Jira project you must have read access on Jira -issues and at least the Maintainer role in the GitLab project that you wish to import into. - -### Jira integration - -This feature uses the existing GitLab [Jira integration](../../../integration/jira/index.md). - -Make sure you have the integration set up before trying to import Jira issues. +- To be able to import issues from a Jira project you must have read access on Jira + issues and at least the Maintainer role in the GitLab project that you wish to import into. +- This feature uses the existing GitLab [Jira integration](../../../integration/jira/index.md). + Make sure you have the integration set up before trying to import Jira issues. ## Import Jira issues to GitLab @@ -59,11 +53,11 @@ Importing large projects may take several minutes depending on the size of the i To import Jira issues to a GitLab project: -1. On the **{issues}** **Issues** page, select **Import Issues** (**{import}**) **> Import from Jira**. +1. On the **{issues}** **Issues** page, select **Actions** (**{ellipsis_v}**) **> Import from Jira**. ![Import issues from Jira button](img/jira/import_issues_from_jira_button_v12_10.png) - The **Import from Jira** option is only visible if you have the [correct permissions](#permissions). + The **Import from Jira** option is only visible if you have the [correct permissions](#prerequisites). The following form appears. If you've previously set up the [Jira integration](../../../integration/jira/index.md), you can now see diff --git a/doc/user/project/import/manifest.md b/doc/user/project/import/manifest.md index 514a6a0cb5a..7b3550d2dc9 100644 --- a/doc/user/project/import/manifest.md +++ b/doc/user/project/import/manifest.md @@ -1,7 +1,6 @@ --- -type: howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -16,12 +15,16 @@ based on a manifest file like the one used by the Use the manifest to import a project with many repositories like the Android Open Source Project (AOSP). -## Requirements +## Prerequisites +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + +- [Manifest import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The Manifest import source is enabled + by default on GitLab.com. - GitLab must use PostgreSQL for its database, because [subgroups](../../group/subgroups/index.md) are needed for the manifest import to work. Read more about the [database requirements](../../../install/requirements.md#database). -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +- At least the Maintainer role on the destination group to import to. ## Manifest format diff --git a/doc/user/project/import/perforce.md b/doc/user/project/import/perforce.md index a96297b1a38..5f06daef0aa 100644 --- a/doc/user/project/import/perforce.md +++ b/doc/user/project/import/perforce.md @@ -1,7 +1,6 @@ --- -type: howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -53,7 +52,7 @@ submit back from Git to Perforce. Here's a few links to get you started: - [`git-p4` manual page](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-p4.html) -- [`git-p4` example usage](https://git.wiki.kernel.org/index.php/Git-p4_Usage) +- [`git-p4` example usage](https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/Git-p4_Usage.html) - [Git book migration guide](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git#_perforce_import) `git p4` and `git filter-branch` are not very good at diff --git a/doc/user/project/import/phabricator.md b/doc/user/project/import/phabricator.md index 8a2dbba1343..7b9615b883c 100644 --- a/doc/user/project/import/phabricator.md +++ b/doc/user/project/import/phabricator.md @@ -1,43 +1,11 @@ --- -type: howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' --- -# Import Phabricator tasks into a GitLab project (deprecated) **(FREE SELF)** +# Import Phabricator tasks into a GitLab project (removed) **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/60562) in GitLab 12.0. - -WARNING: -The Phabricator task importer was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106369) in GitLab 15.7 -and will be removed in GitLab 16.0. - -WARNING: -The Phabricator task importer is in -[beta](../../../policy/alpha-beta-support.md#beta-features) and is -[**not** complete](https://gitlab.com/gitlab-org/gitlab/-/issues/284406). It imports -only an issue's title and description. The GitLab project created during the import -process contains only issues, and the associated repository is disabled. - -GitLab allows you to import all tasks from a Phabricator instance into -GitLab issues. The import creates a single project with the -repository disabled. - -Only the following basic fields are imported: - -- Title -- Description -- State (open or closed) -- Created at -- Closed at - -## Users - -The assignee and author of a user are deducted from a Task's owner and -author: If a user with the same username has access to the namespace -of the project being imported into, then the user is linked. - -## Enable this feature - -Enable Phabricator as an [import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) in the Admin Area. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106369) in GitLab 15.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117649) in 16.0. diff --git a/doc/user/project/import/repo_by_url.md b/doc/user/project/import/repo_by_url.md index 93e3991ba19..4ad51ccb97f 100644 --- a/doc/user/project/import/repo_by_url.md +++ b/doc/user/project/import/repo_by_url.md @@ -1,18 +1,23 @@ --- -type: howto stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Import project from repository by URL **(FREE)** -Prerequisite: +You can import your existing repositories by providing the Git URL. -- At least the Maintainer role on the destination group to import to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +## Prerequisites -You can import your existing repositories by providing the Git URL: +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. + +- [Repository by URL import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) + must be enabled. If not enabled, ask your GitLab administrator to enable it. The Repository by URL import source is enabled + by default on GitLab.com. +- At least the Maintainer role on the destination group to import to. + +## Import project by URL 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. 1. On the right of the page, select **New project**. diff --git a/doc/user/project/import/svn.md b/doc/user/project/import/svn.md deleted file mode 100644 index c9abc0f459d..00000000000 --- a/doc/user/project/import/svn.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: 'index.md#import-from-subversion' -remove_date: '2023-03-15' ---- - -This document was moved to [another location](index.md). - -<!-- This redirect file can be deleted after <2023-03-15>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/import/tfvc.md b/doc/user/project/import/tfvc.md index 0a03513467e..da9d31e0ca8 100644 --- a/doc/user/project/import/tfvc.md +++ b/doc/user/project/import/tfvc.md @@ -1,8 +1,7 @@ --- stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: concepts --- # Migrate from TFVC to Git **(FREE)** diff --git a/doc/user/project/index.md b/doc/user/project/index.md index decf3b071e7..1512039fb25 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- @@ -139,9 +139,8 @@ Prerequisites: - You must have permission to add new projects to a namespace. To check if you have permission: 1. On the top bar, select **Main menu > Groups** and find your group. - 1. Confirm that **New project** is visible in the upper right - corner. Contact your GitLab - administrator if you require permission. + 1. In the upper-right corner, confirm that **New project** is visible. + Contact your GitLab administrator if you require permission. To push your repository and create a project: @@ -180,8 +179,6 @@ Your project's visibility is set to **Private** by default. To change project vi ## Related topics -- For a list of words that you cannot use as project names, see - [reserved project and group names](../../user/reserved_names.md). -- For a list of characters that you cannot use in project and group names, see - [limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names). -- [Manage projects](working_with_projects.md). +- [Reserved project and group names](../../user/reserved_names.md) +- [Limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names) +- [Manage projects](working_with_projects.md) diff --git a/doc/user/project/integrations/apple_app_store.md b/doc/user/project/integrations/apple_app_store.md index 1b41dd0b669..633d565064c 100644 --- a/doc/user/project/integrations/apple_app_store.md +++ b/doc/user/project/integrations/apple_app_store.md @@ -1,19 +1,24 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Apple App Store **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104888) in GitLab 15.8 [with a flag](../../../administration/feature_flags.md) named `apple_app_store_integration`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104888) in GitLab 15.8 [with a flag](../../../administration/feature_flags.md) named `apple_app_store_integration`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/385335) in GitLab 15.10. Feature flag `apple_app_store_integration` removed. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `apple_app_store_integration`. On GitLab.com, this feature is not available. +This feature is part of [Mobile DevOps](../../../ci/mobile_devops.md) developed by [GitLab Incubation Engineering](https://about.gitlab.com/handbook/engineering/incubation/). +The feature is still in development, but you can: -The Apple App Store integration makes it easy to configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com) to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. +- [Request a feature](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=feature_request). +- [Report a bug](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=report_bug). +- [Share feedback](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=general_feedback). -The integration is designed to be able to work out of the box with [fastlane](http://fastlane.tools/), but can be used with other build tools as well. +With the Apple App Store integration, you can configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com) to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. + +The Apple App Store integration works out of the box with [fastlane](https://fastlane.tools/). You can also use this integration with other build tools. ## Prerequisites @@ -29,16 +34,17 @@ GitLab supports enabling the Apple App Store integration at the project level. C 1. Select **Apple App Store**. 1. Turn on the **Active** toggle under **Enable Integration**. 1. Provide the Apple App Store Connect configuration information: - - **Issuer ID**: The Apple App Store Connect Issuer ID can be found in the *Keys* section under *Users and Access* the Apple App Store Connect portal. - - **Key ID**: The Key ID of the new private key that was just generated. - - **Private Key**: The Private Key that was just generated. Note: you are only be able to download this key one time. + - **Issuer ID**: The Apple App Store Connect issuer ID. + - **Key ID**: The key ID of the generated private key. + - **Private Key**: The generated private key. You can download this key only once. 1. Select **Save changes**. After the Apple App Store integration is activated: -- The global variables `$APP_STORE_CONNECT_API_KEY_ISSUER_ID`, `$APP_STORE_CONNECT_API_KEY_KEY_ID`, and `$APP_STORE_CONNECT_API_KEY_KEY` are created for CI/CD use. +- The global variables `$APP_STORE_CONNECT_API_KEY_ISSUER_ID`, `$APP_STORE_CONNECT_API_KEY_KEY_ID`, `$APP_STORE_CONNECT_API_KEY_KEY`, and `$APP_STORE_CONNECT_API_KEY_IS_KEY_CONTENT_BASE64` are created for CI/CD use. - `$APP_STORE_CONNECT_API_KEY_KEY` contains the Base64 encoded Private Key. +- `$APP_STORE_CONNECT_API_KEY_IS_KEY_CONTENT_BASE64` is always `true`. ## Security considerations @@ -48,12 +54,10 @@ Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variab `$APP_STORE_CONNECT_API_KEY_KEY`, and send them to a third-party server. For more details, see [CI/CD variable security](../../../ci/variables/index.md#cicd-variable-security). -## fastlane Example +## Enable the integration in fastlane -Because this integration works out of the box with fastlane adding the code below to an app's `fastlane/Fastfile` activates the integration, and create the connection for any interactions with the Apple App Store uploading a Test Flight or public App Store release. +To enable the integration in fastlane and upload a TestFlight or public App Store release, you can add the following code to your app's `fastlane/Fastfile`: ```ruby -app_store_connect_api_key( - is_key_content_base64: true -) +app_store_connect_api_key ``` diff --git a/doc/user/project/integrations/asana.md b/doc/user/project/integrations/asana.md index 8bc1a984e3d..b1a4d2a2f78 100644 --- a/doc/user/project/integrations/asana.md +++ b/doc/user/project/integrations/asana.md @@ -1,12 +1,12 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Asana **(FREE)** -This integration adds commit messages as comments to Asana tasks. +The Asana integration adds commit messages as comments to Asana tasks. Once enabled, commit messages are checked for Asana task URLs (for example, `https://app.asana.com/0/123456/987654`) or task IDs starting with `#` (for example, `#987654`). Every task ID found gets the commit comment added to it. diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 12039a70d0e..23990036f4e 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -125,7 +125,7 @@ For example: ### Builds not triggered If builds are not triggered, ensure you entered the right GitLab IP address in -Bamboo under **Trigger IP addresses**. Also check [service hook logs](index.md#troubleshooting-integrations) for request failures. +Bamboo under **Trigger IP addresses**. Also check [integration webhook logs](index.md#troubleshooting) for request failures. ### Advanced Atlassian Bamboo features not available in GitLab UI diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 2933203c593..f2af4dc6e4d 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -57,4 +57,4 @@ internal issue tracker, the internal issue is linked. ## Troubleshooting -To see recent service hook deliveries, check [service hook logs](index.md#troubleshooting-integrations). +For recent integration webhook deliveries, check [integration webhook logs](index.md#troubleshooting). diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index 24a2e3d1ebc..b529d728e3b 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/discord_notifications.md b/doc/user/project/integrations/discord_notifications.md index 0163bce3496..60b37f07bb5 100644 --- a/doc/user/project/integrations/discord_notifications.md +++ b/doc/user/project/integrations/discord_notifications.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/emails_on_push.md b/doc/user/project/integrations/emails_on_push.md index c3134e986d3..7f8755f1114 100644 --- a/doc/user/project/integrations/emails_on_push.md +++ b/doc/user/project/integrations/emails_on_push.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/ewm.md b/doc/user/project/integrations/ewm.md index c2371d32853..39dd548e7ca 100644 --- a/doc/user/project/integrations/ewm.md +++ b/doc/user/project/integrations/ewm.md @@ -1,12 +1,12 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Engineering Workflow Management (EWM) **(FREE)** -This integration allows you to go from GitLab to EWM work items mentioned in merge request +The EWM integration allows you to go from GitLab to EWM work items mentioned in merge request descriptions and commit messages. Each work item reference is automatically converted to a link to the work item. diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index 990d0839581..3e75106a9bc 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -1,13 +1,13 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # GitHub **(PREMIUM)** You can update GitHub with pipeline status updates from GitLab. -This integration can help you if you use GitLab for CI/CD. +The GitHub integration can help you if you use GitLab for CI/CD. ![Pipeline status update on GitHub](img/github_status_check_pipeline_update.png) diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 649b0c51818..fabdf07f76f 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -18,6 +18,10 @@ integrations on GitLab.com. ## Installation +In GitLab 15.0 and later, the GitLab for Slack app uses +[granular permissions](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). +Although functionality has not changed, you should [reinstall the app](#update-the-gitlab-for-slack-app). + ### Through the Slack App Directory To enable the GitLab for Slack app for your workspace, @@ -43,8 +47,8 @@ To enable the GitLab integration for your Slack workspace: 1. Select **Install GitLab for Slack app**. 1. Select **Allow** on Slack's confirmation screen. -You can also select **Reinstall GitLab for Slack app** to update the app in your Slack workspace -to the latest version. See [Version history](#version-history) for details. +To update the app in your Slack workspace to the latest version, +you can also select **Reinstall GitLab for Slack app**. ## Slash commands @@ -63,6 +67,7 @@ Replace `<project>` with the project full path, or create a shorter [project ali | `/gitlab <project> issue comment <id> <shift+return> <comment>` | Adds a new comment with the comment body `<comment>` to the issue with the ID `<id>`. | | `/gitlab <project> deploy <from> to <to>` | [Deploys](#the-deploy-slash-command) from the `<from>` environment to the `<to>` environment. | | `/gitlab <project> run <job name> <arguments>` | Executes the [ChatOps](../../../ci/chatops/index.md) job `<job name>` on the default branch. | +| `/gitlab incident declare` | **Beta** Opens modal to [create a new incident](../../../operations/incident_management/slack.md). | ### The deploy slash command @@ -191,8 +196,3 @@ If you're not receiving notifications to a Slack channel, ensure: ### The App Home does not display properly If the [App Home](https://api.slack.com/start/overview#app_home) does not display properly, ensure your [app is up to date](#update-the-gitlab-for-slack-app). - -## Version history - -In GitLab 15.0 and later, the GitLab for Slack app is updated to [Slack's new granular permissions model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). -Although there is no change in functionality, you should [reinstall the app](#update-the-gitlab-for-slack-app). diff --git a/doc/user/project/integrations/google_play.md b/doc/user/project/integrations/google_play.md new file mode 100644 index 00000000000..2ae5a504e06 --- /dev/null +++ b/doc/user/project/integrations/google_play.md @@ -0,0 +1,55 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Google Play **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111621) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `google_play_integration`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/389611) in GitLab 15.11. Feature flag `google_play_integration` removed. + +This feature is part of [Mobile DevOps](../../../ci/mobile_devops.md) developed by [GitLab Incubation Engineering](https://about.gitlab.com/handbook/engineering/incubation/). +The feature is still in development, but you can: + +- [Request a feature](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=feature_request). +- [Report a bug](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=report_bug). +- [Share feedback](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=general_feedback). + +With the Google Play integration, you can configure your CI/CD pipelines to connect to the [Google Play Console](https://play.google.com/console) to build and release apps for Android devices. + +The Google Play integration works out of the box with [fastlane](https://fastlane.tools/). You can also use this integration with other build tools. + +## Enable the integration in GitLab + +Prerequisites: + +- You must have a [Google Play Console](https://play.google.com/console/developers) developer account. +- You must [generate a new service account key for your project](https://developers.google.com/android-publisher/getting_started) from the Google Cloud console. + +To enable the Google Play integration in GitLab: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Google Play**. +1. In **Enable integration**, select the **Active** checkbox. +1. In **Package name**, enter the package name of the app (for example, `com.gitlab.app_name`). +1. In **Service account key (.JSON)**, drag or upload your key file. +1. Select **Save changes**. + +After you enable the integration, the global variables `$SUPPLY_PACKAGE_NAME` and `$SUPPLY_JSON_KEY_DATA` are created for CI/CD use. + +### CI/CD variable security + +Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables, including `$SUPPLY_JSON_KEY_DATA`, and send them to a third-party server. For more information, see [CI/CD variable security](../../../ci/variables/index.md#cicd-variable-security). + +## Enable the integration in fastlane + +To enable the integration in fastlane and upload the build to the given track in Google Play, you can add the following code to your app's `fastlane/Fastfile`: + +```ruby +upload_to_play_store( + track: 'internal', + aab: '../build/app/outputs/bundle/release/app-release.aab' +) +``` diff --git a/doc/user/project/integrations/hangouts_chat.md b/doc/user/project/integrations/hangouts_chat.md index 3537033902d..3470d11b983 100644 --- a/doc/user/project/integrations/hangouts_chat.md +++ b/doc/user/project/integrations/hangouts_chat.md @@ -1,13 +1,13 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Google Chat **(FREE)** -Integrate your project to send notifications from GitLab to a -room of your choice in [Google Chat](https://chat.google.com/) (former Google +You can configure your project to send notifications from GitLab to a +room of your choice in [Google Chat](https://chat.google.com/) (formerly Google Hangouts). ## Integration workflow @@ -28,7 +28,7 @@ notifications to Google Chat: To enable the integration in Google Chat: 1. Enter the room where you want to receive notifications from GitLab. -1. Open the room dropdown list in the upper left and select **Manage webhooks**. +1. In the upper-left corner, from the room dropdown list, select **Manage webhooks**. 1. Enter the name for your webhook, for example "GitLab integration". 1. Optional. Add an avatar for your bot. 1. Select **Save**. diff --git a/doc/user/project/integrations/harbor.md b/doc/user/project/integrations/harbor.md index 596821ba12b..4a586054aee 100644 --- a/doc/user/project/integrations/harbor.md +++ b/doc/user/project/integrations/harbor.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -8,11 +8,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80999) in GitLab 14.9. -Use Harbor as the container registry for your GitLab project. +You can use Harbor as the container registry for your GitLab project. -[Harbor](https://goharbor.io/) is an open source registry that can help you manage artifacts across cloud-native compute platforms, like Kubernetes and Docker. +[Harbor](https://goharbor.io/) is an open-source registry that can help you manage artifacts across cloud-native compute platforms like Kubernetes and Docker. -This integration can help you if you need GitLab CI/CD and a container image repository. +The Harbor integration can help you if you need GitLab CI/CD and a container image repository. ## Prerequisites diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 57947e21736..f7019b2eeb2 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -72,13 +72,14 @@ You can configure the following integrations. | [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | | [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | | [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | -| [Pumble](pumble.md) | Send event notifications to a Pumble channel. | **{dotted-circle}** No | +| [Pumble](pumble.md) | Send event notifications to a Pumble channel. | **{dotted-circle}** No | | Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | | [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | -| [Shimo Workspace](shimo.md) | Use Shimo instead of the GitLab Wiki. | **{dotted-circle}** No | +| [Shimo Workspace](shimo.md) | Use Shimo instead of the GitLab Wiki. | **{dotted-circle}** No | | [GitLab for Slack app](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | | [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | | [Slack slash commands](slack_slash_commands.md) | Enable slash commands in a workspace. | **{dotted-circle}** No | +| [Squash TM](squash_tm.md) | Update Squash TM requirements when GitLab issues are modified. | **{check-circle}** Yes | | [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | | [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | | [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | @@ -102,9 +103,16 @@ supported by `push_hooks` and `tag_push_hooks` events aren't executed. You can change the number of supported branches or tags by changing the [`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). -## Troubleshooting integrations +## Contribute to integrations + +If you're interested in developing a new native integration for GitLab, see: + +- [Integrations development guidelines](../../../development/integrations/index.md) +- [GitLab Developer Portal](https://developer.gitlab.com) + +## Troubleshooting -Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). For more information, see [Troubleshooting webhooks](webhooks.md#troubleshoot-webhooks). +Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). For more information, see [webhook troubleshooting](webhooks.md#troubleshooting). ### `Test Failed. Save Anyway` error @@ -114,7 +122,3 @@ push data to build the test payload, and there are no push events in the project To resolve this error, initialize the repository by pushing a test file to the project and set up the integration again. - -## Contribute to integrations - -To add a new integration, see the [Integrations development guide](../../../development/integrations/index.md). diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index 23525c33e84..df11ed3e57c 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -1,13 +1,13 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # irker (IRC gateway) **(FREE)** -GitLab provides a way to push update messages to an irker server. When -configured, pushes to a project trigger the integration to send data directly +GitLab provides a way to push update messages to an irker server. After you configure +the integration, each push to a project triggers the integration to send data directly to the irker server. See also the [irker integration API documentation](../../../api/integrations.md). diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index fcc364724b7..a50d341c38e 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index 192360f5440..3b5e4030487 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/microsoft_teams.md b/doc/user/project/integrations/microsoft_teams.md index fc5d4d3cc80..5d2c27fc2a4 100644 --- a/doc/user/project/integrations/microsoft_teams.md +++ b/doc/user/project/integrations/microsoft_teams.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -58,4 +58,4 @@ GitLab to send the notifications: ## Related topics -- [Setting up an incoming webhook on Microsoft Teams](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using#setting-up-a-custom-incoming-webhook). +- [Setting up an incoming webhook on Microsoft Teams](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using#setting-up-a-custom-incoming-webhook) diff --git a/doc/user/project/integrations/mlflow_client.md b/doc/user/project/integrations/mlflow_client.md index bd14021ab1c..75fae4647d0 100644 --- a/doc/user/project/integrations/mlflow_client.md +++ b/doc/user/project/integrations/mlflow_client.md @@ -4,33 +4,31 @@ group: Incubation info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. --- -# MLFlow Client Integration **(FREE)** +# MLflow client integration **(FREE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.6 as an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.11 as an [Experiment](../../../policy/alpha-beta-support.md#experiment) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. -DISCLAIMER: -MLFlow Client Integration is an experimental feature being developed by the Incubation Engineering Department, -and will receive significant changes over time. +NOTE: +Model experiment tracking is an [experimental feature](../../../policy/alpha-beta-support.md). +Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. -[MLFlow](https://mlflow.org/) is one of the most popular open source tools for Machine Learning Experiment Tracking. -GitLabs works as a backend to the MLFlow Client, [logging experiments](../ml/experiment_tracking/index.md). +[MLflow](https://mlflow.org/) is a popular open source tool for Machine Learning Experiment Tracking. +GitLab works as a backend to the MLflow Client, [logging experiments](../ml/experiment_tracking/index.md). Setting up your integrations requires minimal changes to existing code. -GitLab plays the role of proxy server, both for artifact storage and tracking data. It reflects the -MLFlow [Scenario 5](https://www.mlflow.org/docs/latest/tracking.html#scenario-5-mlflow-tracking-server-enabled-with-proxied-artifact-storage-access). +GitLab plays the role of a MLflow server. Running `mlflow server` is not necessary. -## Enable MLFlow Client Integration - -Complete this task to enable MLFlow Client Integration. +## Enable MLflow client integration Prerequisites: - A [personal access token](../../../user/profile/personal_access_tokens.md) for the project, with minimum access level of `api`. - The project ID. To find the project ID, on the top bar, select **Main menu > Projects** and find your project. On the left sidebar, select **Settings > General**. -1. Set the tracking URI and token environment variables on the host that runs the code (your local environment, CI pipeline, or remote host). +To enable MLflow client integration: - For example: +1. Set the tracking URI and token environment variables on the host that runs the code. + This can be your local environment, CI pipeline, or remote host. For example: ```shell export MLFLOW_TRACKING_URI="http://<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow" @@ -39,43 +37,66 @@ Prerequisites: 1. If your training code contains the call to `mlflow.set_tracking_uri()`, remove it. -When running the training code, MLFlow will create experiments, runs, log parameters, metrics, +When running the training code, MLflow creates experiments, runs, log parameters, metrics, metadata and artifacts on GitLab. -After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. Runs are registered as Model Candidates, -that can be explored by selecting an experiment. +After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. +Runs are registered as: + +- Model Candidates, which can be explored by selecting an experiment. +- Tags, which are registered as metadata. + +## Associating a candidate to a CI/CD job + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119454) in GitLab 16.1. + +If your training code is being run from a CI/CD job, GitLab can use that information to enhance +candidate metadata. To do so, add the following snippet to your training code within the run +execution context: + +```python +with mlflow.start_run(run_name=f"Candidate {index}"): + # Your training code + + # Start of snippet to be included + if os.getenv('GITLAB_CI'): + mlflow.set_tag('gitlab.CI_JOB_ID', os.getenv('CI_JOB_ID')) + # End of snippet to be included +``` + +## Supported MLflow client methods and caveats + +GitLab supports these methods from the MLflow client. Other methods might be supported but were not +tested. More information can be found in the [MLflow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html). + +| Method | Supported | Version Added | Comments | +|--------------------------|------------------|----------------|----------| +| `get_experiment` | Yes | 15.11 | | +| `get_experiment_by_name` | Yes | 15.11 | | +| `set_experiment` | Yes | 15.11 | | +| `get_run` | Yes | 15.11 | | +| `start_run` | Yes | 15.11 | | +| `log_artifact` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_artifacts` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_batch` | Yes | 15.11 | | +| `log_metric` | Yes | 15.11 | | +| `log_metrics` | Yes | 15.11 | | +| `log_param` | Yes | 15.11 | | +| `log_params` | Yes | 15.11 | | +| `log_figure` | Yes | 15.11 | | +| `log_image` | Yes | 15.11 | | +| `log_text` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `log_dict` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `set_tag` | Yes | 15.11 | | +| `set_tags` | Yes | 15.11 | | +| `set_terminated` | Yes | 15.11 | | +| `end_run` | Yes | 15.11 | | +| `update_run` | Yes | 15.11 | | +| `log_model` | Partial | 15.11 | (15.11) Saves the artifacts, but not the model data. `artifact_path` must be empty. ## Limitations -- The API GitLab supports is the one defined at MLFlow version 1.28.0. +- The API GitLab supports is the one defined at MLflow version 1.28.0. - API endpoints not listed above are not supported. -- During creation of experiments and runs, tags are ExperimentTags and RunTags are stored, even though they are not displayed. -- MLFlow Model Registry is not supported. - -## Supported methods and caveats - -This is a list of methods we support from the MLFlow client. Other methods might be supported but were not -tested. More information can be found in the [MLFlow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html). - -### `set_experiment()` - -Accepts both `experiment_name` and `experiment_id` - -### `start_run()` - -- Nested runs have not been tested. -- `run_name` is not supported - -### `log_param()`, `log_params()`, `log_metric()`, `log_metrics()` - -Work as defined by the documentation - -### `log_artifact()`, `log_artifacts()` - -`artifact_path` must be empty string. - -### `log_model()` - -This is an experimental method in MLFlow, and partial support is offered. It stores the model artifacts, but does -not log the model information. The `artifact_path` parameter must be set to `''`, because Generic Packages do not support folder -structure. +- During creation of experiments and runs, ExperimentTags are stored, even though they are not displayed. +- MLflow Model Registry is not supported. diff --git a/doc/user/project/integrations/mock_ci.md b/doc/user/project/integrations/mock_ci.md index ae1737f8d3f..da09d621d07 100644 --- a/doc/user/project/integrations/mock_ci.md +++ b/doc/user/project/integrations/mock_ci.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -9,12 +9,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: This integration only appears if you're in a [development environment](https://gitlab.com/gitlab-org/gitlab-mock-ci-service#setup-mockci-integration). -To set up the mock CI service server, respond to the following endpoints +To set up the mock CI service server, respond to the following endpoints: - `commit_status`: `#{project.namespace.path}/#{project.path}/status/#{sha}.json` - - Have your service return `200 { status: ['failed'|'canceled'|'running'|'pending'|'success'|'success-with-warnings'|'skipped'|'not_found'] }` - - If the service returns a 404, it is interpreted as `pending` + - Have your service return `200 { status: ['failed'|'canceled'|'running'|'pending'|'success'|'success-with-warnings'|'skipped'|'not_found'] }`. + - If the service returns a 404, the service is interpreted as `pending`. - `build_page`: `#{project.namespace.path}/#{project.path}/status/#{sha}` - - Just where the build is linked to, doesn't matter if implemented + - Where the build is linked to (whether or not it's implemented). -For an example of a mock CI server, see [`gitlab-org/gitlab-mock-ci-service`](https://gitlab.com/gitlab-org/gitlab-mock-ci-service) +For an example of a mock CI server, see [`gitlab-org/gitlab-mock-ci-service`](https://gitlab.com/gitlab-org/gitlab-mock-ci-service). diff --git a/doc/user/project/integrations/pipeline_status_emails.md b/doc/user/project/integrations/pipeline_status_emails.md index 009bb6662ff..9a87b5b3110 100644 --- a/doc/user/project/integrations/pipeline_status_emails.md +++ b/doc/user/project/integrations/pipeline_status_emails.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/pivotal_tracker.md b/doc/user/project/integrations/pivotal_tracker.md index 034be8ab3d8..e1d48037ba8 100644 --- a/doc/user/project/integrations/pivotal_tracker.md +++ b/doc/user/project/integrations/pivotal_tracker.md @@ -1,12 +1,12 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Pivotal Tracker **(FREE)** -This integration adds commit messages as comments to Pivotal Tracker stories. +The Pivotal Tracker integration adds commit messages as comments to Pivotal Tracker stories. Once enabled, commit messages are checked for square brackets containing a hash mark followed by the story ID (for example, `[#555]`). Every story ID found gets the commit comment added to it. diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index cd92e49cada..d17e8e6bfca 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -2,129 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: 'index.md' --- -# Prometheus **(FREE)** +# Prometheus (removed) **(FREE)** -GitLab offers powerful integration with [Prometheus](https://prometheus.io) for -monitoring key metrics of your apps, directly in GitLab. -Metrics for each environment are retrieved from Prometheus, and then displayed -in the GitLab interface. - -![Environment Dashboard](img/prometheus_dashboard.png) - -There are two ways to set up Prometheus integration, depending on where your apps are running: - -- For deployments on Kubernetes, GitLab can be [integrated with an in-cluster Prometheus](#prometheus-cluster-integration) -- For other deployment targets, [specify the Prometheus server](#manual-configuration-of-prometheus). - -Once enabled, GitLab detects metrics from known services in the -[metric library](prometheus_library/index.md). You can also -[add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics) and create -[custom dashboards](../../../operations/metrics/dashboards/index.md). - -## Enabling Prometheus Integration - -### Prometheus cluster integration - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55244) in GitLab 13.11. -> - [Replaced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62725) the Prometheus cluster applications in GitLab 14.0. - -GitLab can query an in-cluster Prometheus for your metrics. -See [Prometheus cluster integration](../../clusters/integrations.md#prometheus-cluster-integration) for details. - -### Manual configuration of Prometheus - -#### Requirements - -Integration with Prometheus requires the following: - -- Prometheus must be configured to collect one of the [supported metrics](prometheus_library/index.md) -- Each metric must have a label to indicate the environment -- GitLab must have network connectivity to the Prometheus server - -#### Getting started - -Installing and configuring Prometheus to monitor applications is fairly straightforward. - -1. [Install Prometheus](https://prometheus.io/docs/prometheus/latest/installation/) -1. Set up one of the [supported monitoring targets](prometheus_library/index.md) -1. Configure the Prometheus server to - [collect their metrics](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) - -#### Configuration in GitLab - -The actual configuration of Prometheus integration in GitLab -requires the domain name or IP address of the Prometheus server you'd like -to integrate with. If the Prometheus resource is secured with Google's Identity-Aware Proxy (IAP), -you can pass information like Client ID and Service Account credentials. -GitLab can use these to access the resource. More information about authentication from a -service account can be found at Google's documentation for -[Authenticating from a service account](https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_service_account). - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. -1. Select **Prometheus**. -1. For **API URL**, provide the domain name or IP address of your server, such as - `http://prometheus.example.com/` or `http://192.0.2.1/`. -1. (Optional) In **Google IAP Audience Client ID**, provide the Client ID of the - Prometheus OAuth Client secured with Google IAP. -1. (Optional) In **Google IAP Service Account JSON**, provide the contents of the - Service Account credentials file that is authorized to access the Prometheus resource. - The JSON key `token_credential_uri` is discarded to prevent - [Server-side Request Forgery (SSRF)](https://www.hackerone.com/application-security/how-server-side-request-forgery-ssrf). -1. Select **Save changes**. - -![Configure Prometheus Service](img/prometheus_manual_configuration_v13_2.png) - -#### Thanos configuration in GitLab - -You can configure [Thanos](https://thanos.io/) as a drop-in replacement for Prometheus -with GitLab. Use the domain name or IP address of the Thanos server you'd like -to integrate with. - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. -1. Select **Prometheus**. -1. Provide the domain name or IP address of your server, for example - `http://thanos.example.com/` or `http://192.0.2.1/`. -1. Select **Save changes**. - -### Precedence with multiple Prometheus configurations - -Although you can enable both a [manual configuration](#manual-configuration-of-prometheus) -and [cluster integration](#prometheus-cluster-integration) of Prometheus, you -can use only one: - -- If you have enabled a - [Prometheus manual configuration](#manual-configuration-of-prometheus) - and a [Prometheus cluster integration](#prometheus-cluster-integration), - the manual configuration takes precedence and is used to run queries from - [custom dashboards](../../../operations/metrics/dashboards/index.md) and - [custom metrics](../../../operations/metrics/index.md#adding-custom-metrics). -- If you have managed Prometheus applications installed on Kubernetes clusters - at **different** levels (project, group, instance), the order of precedence is described in - [Cluster precedence](../../instance/clusters/index.md#cluster-precedence). -- If you have managed Prometheus applications installed on multiple Kubernetes - clusters at the **same** level, the Prometheus application of a cluster with a - matching [environment scope](../../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) is used. - -## Determining the performance impact of a merge - -Developers can view the performance impact of their changes in the merge -request workflow. This feature requires [Kubernetes](prometheus_library/kubernetes.md) metrics. - -When a source branch has been deployed to an environment, a sparkline and -numeric comparison of the average memory consumption displays. On the -sparkline, a dot indicates when the current changes were deployed, with up to 30 minutes of -performance data displayed before and after. The comparison shows the difference -between the 30 minute average before and after the deployment. This information -is updated after each commit has been deployed. - -Once merged and the target branch has been redeployed, the metrics switches -to show the new environments this revision has been deployed to. - -Performance data is available for the duration it is persisted on the -Prometheus server. - -![Merge Request with Performance Impact](img/merge_request_performance.png) +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index 1e9319fa7c7..848ccbb85f6 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -2,48 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring AWS resources (DEPRECATED) **(FREE)** +# Monitoring AWS resources (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab supports automatically detecting and monitoring AWS resources, starting -with the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/) (ELB). -This is provided by leveraging the official [Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter), which translates [Cloudwatch metrics](https://aws.amazon.com/cloudwatch/) into -a Prometheus readable form. - -## Requirements - -You must enable the [Prometheus service](../prometheus.md). - -## Supported metrics - -| Name | Query | -|----------------------|-------| -| Throughput (req/sec) | `sum(aws_elb_request_count_sum{%{environment_filter}}) / 60` | -| Latency (ms) | `avg(aws_elb_latency_average{%{environment_filter}}) * 1000` | -| HTTP Error Rate (%) | `sum(aws_elb_httpcode_backend_5_xx_sum{%{environment_filter}}) / sum(aws_elb_request_count_sum{%{environment_filter}})` | - -## Configuring Prometheus to monitor for Cloudwatch metrics - -To get started with Cloudwatch monitoring, install and configure the -[Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter). The -Cloudwatch exporter retrieves and parses the specified Cloudwatch metrics, and -translates them into a Prometheus monitoring endpoint. - -The only supported AWS resource is the Elastic Load Balancer, whose Cloudwatch -metrics are listed in [this AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-cloudwatch-metrics.html). - -You can [download a sample Cloudwatch Exporter configuration file](../samples/cloudwatch.yml) -that's configured for basic AWS ELB monitoring. - -## Specifying the Environment label - -To isolate and display only the relevant metrics for a given environment, -GitLab needs a method to detect which labels are associated. To do this, GitLab -[looks for an `environment` label](index.md#identifying-environments). +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index b4533d83acd..ca015d7dc8a 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -2,34 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring HAProxy (DEPRECATED) **(FREE)** +# Monitoring HAProxy (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab has support for automatically detecting and monitoring HAProxy. This is provided by leveraging the [HAProxy Exporter](https://github.com/prometheus/haproxy_exporter), which translates HAProxy statistics into a Prometheus readable form. - -## Requirements - -The [Prometheus service](../prometheus.md) must be enabled. - -## Metrics supported - -| Name | Query | -| ---- | ----- | -| Throughput (req/sec) | `sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) by (code)` | -| HTTP Error Rate (%) | `sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m]))` | - -## Configuring Prometheus to monitor for HAProxy metrics - -To get started with NGINX monitoring, you should install and configure the [HAProxy exporter](https://github.com/prometheus/haproxy_exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. - -## Specifying the Environment label - -In order to isolate and only display relevant metrics for a given environment -however, GitLab needs a method to detect which labels are associated. To do this, GitLab [looks for an `environment` label](index.md#identifying-environments). +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/index.md b/doc/user/project/integrations/prometheus_library/index.md index afefe80271e..ee0b0d1fccb 100644 --- a/doc/user/project/integrations/prometheus_library/index.md +++ b/doc/user/project/integrations/prometheus_library/index.md @@ -2,43 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Prometheus Metrics library (DEPRECATED) **(FREE)** +# Prometheus Metrics library (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). - -## Exporters - -Currently supported exporters are: - -- [Kubernetes](kubernetes.md) -- [NGINX](nginx.md) -- [NGINX Ingress Controller 0.9.0-0.15.x](nginx_ingress_vts.md) -- [NGINX Ingress Controller 0.16.0+](nginx_ingress.md) -- [HAProxy](haproxy.md) -- [Amazon Cloud Watch](cloudwatch.md) - -We have tried to surface the most important metrics for each exporter, and -continue to add support for additional exporters in future releases. If you -would like to add support for other official exporters, contributions are welcome. - -## Identifying Environments - -GitLab retrieves performance data from the configured Prometheus server, and -attempts to identifying the presence of known metrics. Once identified, GitLab -then needs to be able to map the data to a particular environment. - -To isolate and only display relevant metrics for a given environment, -GitLab needs a method to detect which labels are associated. To do that, -GitLab uses the defined queries and fills in the environment specific variables. -Typically this involves looking for the -[`$CI_ENVIRONMENT_SLUG`](../../../../ci/variables/index.md#predefined-cicd-variables), -but may also include other information such as the project's Kubernetes namespace. -Each search query is defined in the [exporter specific documentation](#exporters). +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 34a6823f007..4b8b9b4bc21 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -2,66 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring Kubernetes (DEPRECATED) **(FREE)** +# Monitoring Kubernetes (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab has support for automatically detecting and monitoring Kubernetes metrics. - -## Requirements - -The [Prometheus](../prometheus.md) and [Kubernetes](../../../infrastructure/clusters/index.md) -integration services must be enabled. - -## Metrics supported - -- Average Memory Usage (MB): - - ```prometheus - avg(sum(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) by (job)) without (job) / count(avg(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) without (job)) /1024/1024 - ``` - -- Average CPU Utilization (%): - - ```prometheus - avg(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}[15m])) by (job)) without (job) / count(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}[15m])) by (pod_name)) - ``` - -## Configuring Prometheus to monitor for Kubernetes metrics - -Prometheus needs to be deployed into the cluster and configured properly to gather Kubernetes metrics. GitLab supports two methods for doing so: - -- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [query a Prometheus in a connected cluster](../../../clusters/integrations.md#prometheus-cluster-integration). The in-cluster Prometheus can be configured to automatically collect application metrics from your cluster. -- To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/). - -## Specifying the Environment - -To isolate and only display relevant CPU and Memory metrics for a given environment, GitLab needs a method to detect which containers it is running. Because these metrics are tracked at the container level, traditional Kubernetes labels are not available. - -Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name should begin with [CI_ENVIRONMENT_SLUG](../../../../ci/variables/index.md#predefined-cicd-variables). It can be followed by a `-` and additional content if desired. For example, a deployment name of `review-homepage-5620p5` would match the `review/homepage` environment. - -## Displaying Canary metrics **(PREMIUM)** - -GitLab also gathers Kubernetes metrics for [canary deployments](../../canary_deployments.md), allowing easy comparison between the current deployed version and the canary. - -These metrics expect the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name to begin with `$CI_ENVIRONMENT_SLUG-canary`, to isolate the canary metrics. - -### Canary metrics supported - -- Average Memory Usage (MB) - - ```prometheus - avg(sum(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-canary-(.*)",namespace="%{kube_namespace}"}) by (job)) without (job) / count(avg(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-canary-(.*)",namespace="%{kube_namespace}"}) without (job)) /1024/1024 - ``` - -- Average CPU Utilization (%) - - ```prometheus - avg(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-canary-(.*)",namespace="%{kube_namespace}"}[15m])) by (job)) without (job) / count(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-canary-(.*)",namespace="%{kube_namespace}"}[15m])) by (pod_name)) - ``` +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index f0a3b25f11a..995de6a28f3 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -2,42 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring NGINX (DEPRECATED) **(FREE)** +# Monitoring NGINX (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab has support for automatically detecting and monitoring NGINX. This is provided by leveraging the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates [VTS statistics](https://github.com/vozlt/nginx-module-vts) into a Prometheus readable form. - -## Requirements - -The [Prometheus service](../prometheus.md) must be enabled. - -## Metrics supported - -NGINX server metrics are detected, which tracks the pages and content directly served by NGINX. - -[`environment_filter`](../../../../operations/metrics/dashboards/variables.md#environment_filter) is one of the predefined variables that metrics dashboards support. - -| Name | Query | -| ---- | ----- | -| Throughput (req/sec) | `sum(rate(nginx_server_requests{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (code)` | -| Latency (ms) | `avg(nginx_server_requestMsec{%{environment_filter}})` | -| HTTP Error Rate (HTTP Errors / sec) | `sum(rate(nginx_server_requests{code="5xx", %{environment_filter}}[2m]))` | -| HTTP Error (%)| `sum(rate(nginx_server_requests{code=~"5.*", host="*", %{environment_filter}}[2m])) / sum(rate(nginx_server_requests{code="total", host="*", %{environment_filter}}[2m])) * 100` | - -## Configuring Prometheus to monitor for NGINX metrics - -To get started with NGINX monitoring, you should first enable the [VTS statistics](https://github.com/vozlt/nginx-module-vts) module for your NGINX server. This captures and displays statistics in an HTML readable form. Next, you should install and configure the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. - -If you are using NGINX as your Kubernetes Ingress, GitLab [automatically detects](nginx_ingress.md) the metrics once enabled in 0.9.0 and later releases. - -## Specifying the Environment label - -In order to isolate and only display relevant metrics for a given environment -however, GitLab needs a method to detect which labels are associated. To do this, GitLab [looks for an `environment` label](index.md#identifying-environments). +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 947210541f4..03f88e6f7c5 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -2,48 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring NGINX Ingress Controller (DEPRECATED) **(FREE)** +# Monitoring NGINX Ingress Controller (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the built-in Prometheus metrics included with Kubernetes NGINX Ingress controller [version 0.16.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0160) onward. - -NOTE: -NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. - -## Requirements - -[Prometheus integration](../prometheus.md) must be active. - -## Metrics supported - -| Name | Query | -| ---- | ----- | -| Throughput (req/sec) | `sum(label_replace(rate(nginx_ingress_controller_requests{namespace="%{kube_namespace}",ingress=~".*%{ci_environment_slug}.*"}[2m]), "status_code", "${1}xx", "status", "(.)..")) by (status_code)` | -| Latency (ms) | `sum(rate(nginx_ingress_controller_ingress_upstream_latency_seconds_sum{namespace="%{kube_namespace}",ingress=~".*%{ci_environment_slug}.*"}[2m])) / sum(rate(nginx_ingress_controller_ingress_upstream_latency_seconds_count{namespace="%{kube_namespace}",ingress=~".*%{ci_environment_slug}.*"}[2m])) * 1000` | -| HTTP Error Rate (%) | `sum(rate(nginx_ingress_controller_requests{status=~"5.*",namespace="%{kube_namespace}",ingress=~".*%{ci_environment_slug}.*"}[2m])) / sum(rate(nginx_ingress_controller_requests{namespace="%{kube_namespace}",ingress=~".*%{ci_environment_slug}.*"}[2m])) * 100` | - -## Configuring NGINX Ingress monitoring - -Version 0.9.0 and above of [NGINX Ingress](https://github.com/kubernetes/ingress-nginx) have built-in support for exporting Prometheus metrics. To enable, a ConfigMap setting must be passed: `enable-vts-status: "true"`. Once enabled, a Prometheus metrics endpoint starts running on port 10254. - -Next, the Ingress needs to be annotated for Prometheus monitoring. Two new annotations need to be added: - -- `prometheus.io/scrape: "true"` -- `prometheus.io/port: "10254"` - -Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible to edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard). - -## Specifying the Environment label - -To isolate and display only relevant metrics for a given environment, GitLab needs a method to -detect which labels are associated. To do this, GitLab searches for metrics with appropriate labels. -In this case, the `ingress` label must include the value `<CI_ENVIRONMENT_SLUG>`. - -If you have used [Auto Deploy](../../../../topics/autodevops/stages.md#auto-deploy) to deploy your app, this format is used automatically and metrics are detected with no action on your part. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md index f8057866592..837be9d1e0a 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md @@ -2,46 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: '../../../../operations/index.md' --- -# Monitoring NGINX Ingress Controller with VTS metrics (DEPRECATED) **(FREE)** +# Monitoring NGINX Ingress Controller with VTS metrics (removed) **(FREE)** -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) -in GitLab 14.7, and is planned for removal in GitLab 16.0. - -NOTE: -[NGINX Ingress version 0.16](nginx_ingress.md) and above have built-in Prometheus metrics, which are different than the VTS based metrics. - -GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the included VTS Prometheus metrics exporter in [version 0.9.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#09-beta1) through [0.15.x](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0150). - -## Requirements - -[Prometheus integration](../prometheus.md) must be active. - -## Metrics supported - -| Name | Query | -| ---- | ----- | -| Throughput (req/sec) | `sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code)` | -| Latency (ms) | `avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"})` | -| HTTP Error Rate (%) | `sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) / sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) * 100` | - -## Configuring NGINX Ingress monitoring - -Version 0.9.0 and above of [NGINX Ingress](https://github.com/kubernetes/ingress-nginx) has built-in support for exporting Prometheus metrics. To enable, a ConfigMap setting must be passed: `enable-vts-status: "true"`. Once enabled, a Prometheus metrics endpoint begins running on port 10254. - -Next, the Ingress needs to be annotated for Prometheus monitoring. Two new annotations need to be added: - -- `prometheus.io/scrape: "true"` -- `prometheus.io/port: "10254"` - -Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard). - -## Specifying the Environment label - -To isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do this, GitLab searches for metrics with appropriate labels. In this case, the `upstream` label must be of the form `<KUBE_NAMESPACE>-<CI_ENVIRONMENT_SLUG>-*`. - -If you have used [Auto Deploy](../../../../topics/autodevops/stages.md#auto-deploy) to deploy your app, this format is used automatically and metrics are detected with no action on your part. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/user/project/integrations/pumble.md b/doc/user/project/integrations/pumble.md index f9c0c79be1b..e05ff9489ca 100644 --- a/doc/user/project/integrations/pumble.md +++ b/doc/user/project/integrations/pumble.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 0a399d3481f..df8a6681e2b 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -1,13 +1,12 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Redmine **(FREE)** -Use [Redmine](https://www.redmine.org/) as the issue tracker. - +You can use [Redmine](https://www.redmine.org/) as an external issue tracker. To enable the Redmine integration in a project: 1. On the top bar, select **Main menu > Projects** and find your project. diff --git a/doc/user/project/integrations/servicenow.md b/doc/user/project/integrations/servicenow.md index a34655c8e35..a0beb71d83f 100644 --- a/doc/user/project/integrations/servicenow.md +++ b/doc/user/project/integrations/servicenow.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -9,6 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w ServiceNow offers several integrations to help centralize and automate your management of GitLab workflows. +To simplify your stack and streamline your processes, you should use GitLab [deployment approvals](../../../ci/environments/deployment_approvals.md) whenever possible. + ## GitLab spoke With the GitLab spoke in ServiceNow, you can automate actions for GitLab diff --git a/doc/user/project/integrations/shimo.md b/doc/user/project/integrations/shimo.md index ca09de06dc6..cf9745929a1 100644 --- a/doc/user/project/integrations/shimo.md +++ b/doc/user/project/integrations/shimo.md @@ -1,10 +1,10 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2023-05-22' --> +<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> # Shimo (deprecated) **(FREE)** diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 09e7189d4f5..5b769b84663 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -1,16 +1,18 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> -# Slack notifications **(FREE)** +# Slack notifications (deprecated) **(FREE)** WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/372411) on GitLab.com -in GitLab 15.9 and is [planned for removal](https://gitlab.com/groups/gitlab-org/-/epics/8673). -For GitLab.com, use the [GitLab for Slack app](gitlab_slack_application.md) instead. -For self-managed GitLab instances, you can continue to use this feature. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/372411) in GitLab 15.9 +and is planned for removal in 17.0. Use the [GitLab for Slack app](gitlab_slack_application.md) instead. +This change is a breaking change. +For the planned support of the GitLab for Slack app for self-managed instances, +see [epic 1211](https://gitlab.com/groups/gitlab-org/-/epics/1211). The Slack notifications integration enables your GitLab project to send events (such as issue creation) to your existing Slack team as notifications. Setting up @@ -125,7 +127,7 @@ To view which of these problems is the cause of the issue: ``` If GitLab does not trust HTTPS connections to itself, -[add your certificate to the GitLab trusted certificates](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). +[add your certificate to the GitLab trusted certificates](https://docs.gitlab.com/omnibus/settings/ssl/index.html#install-custom-public-certificates). If GitLab does not trust connections to Slack, the GitLab OpenSSL trust store is incorrect. Typical causes are: @@ -150,3 +152,5 @@ p.each do |project| project.slack_integration.update!(:active, false) end ``` + +<!--- end_remove --> diff --git a/doc/user/project/integrations/slack_slash_commands.md b/doc/user/project/integrations/slack_slash_commands.md index 2563cec2b05..4fa4b75422e 100644 --- a/doc/user/project/integrations/slack_slash_commands.md +++ b/doc/user/project/integrations/slack_slash_commands.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -15,7 +15,7 @@ working in Slack, you can use Slack slash commands. To use Slack slash commands, you must configure both Slack and GitLab. GitLab can also send events (for example, `issue created`) to Slack as notifications. -The [Slack notifications service](slack.md) is configured separately. +The [Slack notifications integration](slack.md) is configured separately. ## Configure GitLab and Slack diff --git a/doc/user/project/integrations/squash_tm.md b/doc/user/project/integrations/squash_tm.md new file mode 100644 index 00000000000..2a1106edb0c --- /dev/null +++ b/doc/user/project/integrations/squash_tm.md @@ -0,0 +1,37 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Squash TM **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337855) in GitLab 15.10. + +When [Squash TM](https://www.squashtest.com/squash-gitlab-integration?lang=en) (Test Management) +integration is enabled and configured in GitLab, issues (typically user stories) created in GitLab +are synchronized as requirements in Squash TM and test progress is reported in GitLab issues. + +## Configure Squash TM + +1. Optional. Ask your system administrator to [configure a token in the properties file](https://tm-en.doc.squashtest.com/latest/redirect/gitlab-integration-token.html). +1. Follow the [Squash TM documentation](https://tm-en.doc.squashtest.com/latest/redirect/gitlab-integration-configuration.html) to: + 1. Create a GitLab server. + 1. Enable the `Xsquash4GitLab` plugin + 1. Configure a synchronization. + 1. From the **Real-time synchronization** panel, copy the following fields to use later in GitLab: + + - **Webhook URL**. + - **Secret token** if your Squash TM system administrator configured one at step 1. + +## Configure GitLab + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Squash TM**. +1. Ensure that the **Active** toggle is enabled. +1. In the **Trigger** section, indicate which type of issue is concerned by the real-time synchronization. +1. Complete the fields: + + - Enter the **Squash TM webhook URL**, + - Enter the **secret token** if your Squash TM system administrator configured it earlier. diff --git a/doc/user/project/integrations/unify_circuit.md b/doc/user/project/integrations/unify_circuit.md index d465b4e50f0..d20b19a3aaa 100644 --- a/doc/user/project/integrations/unify_circuit.md +++ b/doc/user/project/integrations/unify_circuit.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w The Unify Circuit integration sends notifications from GitLab to a Circuit conversation. -## Set up Unify Circuit service +## Set up Unify Circuit In Unify Circuit, [add a webhook](https://www.circuit.com/unifyportalfaqdetail?articleId=164448) and copy its URL. diff --git a/doc/user/project/integrations/webex_teams.md b/doc/user/project/integrations/webex_teams.md index 233209966aa..c755c7a5c17 100644 --- a/doc/user/project/integrations/webex_teams.md +++ b/doc/user/project/integrations/webex_teams.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md index 53177004888..fda778aa167 100644 --- a/doc/user/project/integrations/webhook_events.md +++ b/doc/user/project/integrations/webhook_events.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -271,6 +271,7 @@ Payload example: "human_time_estimate": null, "human_time_change": null, "weight": null, + "health_status": "at_risk", "iid": 23, "url": "http://example.com/diaspora/issues/23", "state": "opened", @@ -861,6 +862,7 @@ Payload example: "visibility_level":20, "path_with_namespace":"gitlabhq/gitlab-test", "default_branch":"master", + "ci_config_path":"", "homepage":"http://example.com/gitlabhq/gitlab-test", "url":"http://example.com/gitlabhq/gitlab-test.git", "ssh_url":"git@example.com:gitlabhq/gitlab-test.git", @@ -885,7 +887,10 @@ Payload example: "title": "MS-Viewport", "created_at": "2013-12-03T17:23:34Z", "updated_at": "2013-12-03T17:23:34Z", + "last_edited_at": "2013-12-03T17:23:34Z", + "last_edited_by_id": 1, "milestone_id": null, + "state_id": 1, "state": "opened", "blocking_discussions_resolved": true, "work_in_progress": false, @@ -893,6 +898,11 @@ Payload example: "merge_status": "unchecked", "target_project_id": 14, "description": "", + "total_time_spent": 1800, + "time_change": 30, + "human_total_time_spent": "30m", + "human_time_change": "30s", + "human_time_estimate": "30m", "url": "http://example.com/diaspora/merge_requests/1", "source": { "name":"Awesome Project", @@ -929,6 +939,7 @@ Payload example: "last_commit": { "id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "message": "fixed readme", + "title": "Update file README.md", "timestamp": "2012-01-03T23:36:29+02:00", "url": "http://example.com/awesome_space/awesome_project/commits/da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "author": { @@ -997,7 +1008,15 @@ Payload example: "type": "ProjectLabel", "group_id": 41 }] - } + }, + "last_edited_at": { + "previous": null, + "current": "2023-03-15 00:00:10 UTC" + }, + "last_edited_by_id": { + "previous": null, + "current": 3278533 + }, }, "assignees": [ { @@ -1074,7 +1093,8 @@ Payload example: "message": "adding an awesome page to the wiki", "slug": "awesome", "url": "http://example.com/root/awesome-project/-/wikis/awesome", - "action": "create" + "action": "create", + "diff_url": "http://example.com/root/awesome-project/-/wikis/home/diff?version_id=78ee4a6705abfbff4f4132c6646dbaae9c8fb6ec" } } ``` @@ -1392,6 +1412,7 @@ Payload example: "build_started_at": null, "build_finished_at": null, "build_duration": null, + "build_queued_duration": 1095.588715, // duration in seconds "build_allow_failure": false, "build_failure_reason": "script_failure", "retries_count": 2, // the second retry of this job @@ -1487,6 +1508,7 @@ Payload example: "deployable_id": 796, "deployable_url": "http://10.126.0.2:3000/root/test-deployment-webhooks/-/jobs/796", "environment": "staging", + "environment_tier": "staging", "environment_slug": "staging", "environment_external_url": "https://staging.example.com", "project": { diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 3d971af5c2a..7fcb2680504 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -99,7 +99,7 @@ You can define URL variables directly using the REST API. ## Configure your webhook receiver endpoint Webhook receiver endpoints should be fast and stable. -Slow and unstable receivers can be [disabled automatically](#failing-webhooks) to ensure system reliability. Webhooks that fail can lead to retries, [which cause duplicate events](#webhook-fails-or-multiple-webhook-requests-are-triggered). +Slow and unstable receivers can be [disabled automatically](#failing-webhooks) to ensure system reliability. Webhooks that fail might lead to [duplicate events](#webhook-fails-or-multiple-webhook-requests-are-triggered). Endpoints should follow these best practices: @@ -118,24 +118,28 @@ Endpoints should follow these best practices: - Never return `500` server error status responses if the event has been handled as this can cause the webhook to be [temporarily disabled](#failing-webhooks). - Invalid HTTP responses are treated as failed requests. -### Failing webhooks +## Failing webhooks -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60837) in GitLab 13.12 [with a flag](../../../administration/feature_flags.md) named `web_hooks_disable_failed`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/329849) in GitLab 15.7. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/329849) in GitLab 15.7. Feature flag `web_hooks_disable_failed` removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60837) for project webhooks in GitLab 13.12 [with a flag](../../../administration/feature_flags.md) named `web_hooks_disable_failed`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/329849) for project webhooks in GitLab 15.7. Feature flag `web_hooks_disable_failed` removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385902) for group webhooks in GitLab 15.10. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/390157) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. -If a webhook fails repeatedly, it may be disabled automatically. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. +On GitLab.com, this feature is available. -Webhooks that return response codes in the `5xx` range are understood to be failing +Project or group webhooks that fail four consecutive times are automatically disabled. + +Project or group webhooks that return response codes in the `5xx` range are understood to be failing intermittently and are temporarily disabled. These webhooks are initially disabled -for 1 minute, which is extended on each retry up to a maximum of 24 hours. +for one minute, which is extended on each subsequent failure up to a maximum of 24 hours. -Webhooks that return response codes in the `4xx` range are understood to be +Project or group webhooks that return response codes in the `4xx` range are understood to be misconfigured and are permanently disabled until you manually re-enable them yourself. -See [Troubleshooting](#troubleshoot-webhooks) for more information on -disabled webhooks and how to re-enable them. +For more information about disabled webhooks, see [troubleshooting](#troubleshooting). ## Test a webhook @@ -146,7 +150,7 @@ For example, to test `push events`, your project should have at least one commit NOTE: Testing is not supported for some types of events for project and groups webhooks. -Read more in [issue 379201](https://gitlab.com/gitlab-org/gitlab/-/issues/379201). +For more information, see [issue 379201](https://gitlab.com/gitlab-org/gitlab/-/issues/379201). Prerequisites: @@ -254,7 +258,7 @@ Image URLs are not rewritten if: ## Events -For more information about supported events for Webhooks, go to [Webhook events](webhook_events.md). +For more information about supported events for webhooks, see [webhook events](webhook_events.md). ## Delivery headers @@ -270,7 +274,7 @@ Webhook requests to your endpoint include the following headers: | `X-Gitlab-Event` | Name of the webhook type. Corresponds to [event types](webhook_events.md) but in the format `"<EVENT> Hook"`. | `"Push Hook"` | | `X-Gitlab-Event-UUID` | Unique ID per webhook that is not recursive. A hook is recursive if triggered by an earlier webhook that hit the GitLab instance. Recursive webhooks have the same value for this header. | `"13792a34-cac6-4fda-95a8-c58e00a3954e"` | -## Troubleshoot webhooks +## Troubleshooting > **Recent events** for group webhooks [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325642) in GitLab 15.3. @@ -288,8 +292,9 @@ To view the table: 1. Scroll down to the webhooks. 1. Each [failing webhook](#failing-webhooks) has a badge listing it as: - - **Failed to connect** if it is misconfigured, and needs manual intervention to re-enable it. - - **Fails to connect** if it is temporarily disabled and is retrying later. + - **Failed to connect** if it's misconfigured and must be manually re-enabled. + - **Fails to connect** if it's temporarily disabled and is automatically + re-enabled after the timeout limit has elapsed. ![Badges on failing webhooks](img/failed_badges.png) @@ -325,8 +330,7 @@ Missing intermediate certificates are common causes of verification failure. ### Webhook fails or multiple webhook requests are triggered -If you are receiving multiple webhook requests, the webhook might have timed out and -been retried. +If you're receiving multiple webhook requests, the webhook might have timed out. GitLab expects a response in [10 seconds](../../../user/gitlab_com/index.md#other-limits). On self-managed GitLab instances, you can [change the webhook timeout limit](../../../administration/instance_limits.md#webhook-timeout). diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index a020cc61533..ee5ca8eca78 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/integrations/zentao.md b/doc/user/project/integrations/zentao.md index ca25c1214b7..19f6a3e315c 100644 --- a/doc/user/project/integrations/zentao.md +++ b/doc/user/project/integrations/zentao.md @@ -1,10 +1,10 @@ --- stage: Manage -group: Integrations +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2023-05-22' --> +<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> # ZenTao (deprecated) **(PREMIUM)** diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 234faa893eb..f5d0382ccd8 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -53,7 +53,7 @@ the issue board feature. > - Multiple issue boards per project [moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53811) to GitLab Free in 12.1. > - Multiple issue boards per group are available in GitLab Premium. -Multiple issue boards allow for more than one issue board for a given project in GitLab Free or group in GitLab Premium and higher tiers. +Multiple issue boards allow for more than one issue board for a given project in the Free tier or group in the Premium and Ultimate tier. This is great for large projects with more than one team or when a repository hosts the code of multiple products. Using the search box at the top of the menu, you can filter the listed boards. @@ -74,7 +74,7 @@ Prerequisites: To create a new issue board: -1. Select the dropdown list with the current board name in the upper left corner of the issue boards page. +1. In the upper-left corner of the issue board page, select the dropdown list with the current board name. 1. Select **Create new board**. 1. Enter the new board's name and select its scope: milestone, labels, assignee, or weight. @@ -86,7 +86,7 @@ Prerequisites: To delete the currently active issue board: -1. Select the dropdown list with the current board name in the upper left corner of the issue boards page. +1. In the upper-left corner of the issue board page, select the dropdown list with the current board name. 1. Select **Delete board**. 1. Select **Delete** to confirm. @@ -97,9 +97,9 @@ Here are some common use cases for issue boards. For examples of using issue boards along with [epics](../group/epics/index.md), [issue health status](issues/managing_issues.md#health-status), and -[scoped labels](labels.md#scoped-labels) for various Agile frameworks, check: +[scoped labels](labels.md#scoped-labels) for various Agile frameworks, see: -- The [How to use GitLab for Agile portfolio planning and project management](https://about.gitlab.com/blog/2020/11/11/gitlab-for-agile-portfolio-planning-project-management/) blog post (November 2020) +- [How to use GitLab for Agile portfolio planning and project management](https://about.gitlab.com/blog/2020/11/11/gitlab-for-agile-portfolio-planning-project-management/) blog post (November 2020) - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Cross-project Agile work management with GitLab](https://www.youtube.com/watch?v=5J0bonGoECs) (15 min, July 2020) @@ -132,10 +132,17 @@ If you have the labels **Backend**, **Frontend**, **Staging**, and With [multiple issue boards](#multiple-issue-boards), each team can have their own board to organize their workflow individually. +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=2IaMXteKSD4">Portfolio Planning - Portfolio Management</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/2IaMXteKSD4" frameborder="0" allowfullscreen> </iframe> +</figure> + #### Scrum team With multiple issue boards, each team has one board. Now you can move issues through each -part of the process. For instance: **To Do**, **Doing**, and **Done**. +part of the process. For example: **To Do**, **Doing**, and **Done**. #### Organization of topics @@ -143,7 +150,7 @@ Create lists to order issues by topic and quickly change them between topics or such as between **UX**, **Frontend**, and **Backend**. The changes are reflected across boards, as changing lists updates the labels on each issue accordingly. -#### Advanced team handover +#### Issue board workflow between teams For example, suppose we have a UX team with an issue board that contains: @@ -160,6 +167,9 @@ When finished with something, they move the card to **Frontend**. The Frontend t Cards finished by the UX team automatically appear in the **Frontend** column when they are ready for them. +For a tutorial how to set up your boards in a similar way with [scoped labels](labels.md#scoped-labels), see +[Tutorial: Set up issue boards for team hand-off](../../tutorials/boards_for_teams/index.md). + NOTE: For a broader use case, see the blog post [What is GitLab Flow?](https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/). @@ -207,7 +217,7 @@ Prerequisites: - You must have at least the Reporter role for the project. When an issue is created, the system assigns a relative order value that is greater than the maximum value -of that issue's project or root group. This means the issue will be at the bottom of any issue list that +of that issue's project or root group. This means the issue is at the bottom of any issue list that it appears in. When you visit a board, issues appear ordered in any list. You're able to change @@ -226,6 +236,18 @@ This ordering also affects [issue lists](issues/sorting_issue_lists.md). Changing the order in an issue board changes the ordering in an issue list, and vice versa. +## Focus mode + +To enable or disable focus mode, in the upper-right corner, select **Toggle focus mode** (**{maximize}**). +In focus mode, the navigation UI is hidden, allowing you to focus on issues in the board. + +## Group issue boards + +Accessible at the group navigation level, a group issue board offers the same features as a project-level board. +It can display issues from all projects that fall under the group and its descendant subgroups. + +Users on GitLab Free can use a single group issue board. + ## GitLab Enterprise features for issue boards GitLab issue boards are available on the GitLab Free tier, but some @@ -245,7 +267,7 @@ This allows you to create unique boards according to your team's need. You can define the scope of your board when creating it or by selecting the **Edit board** button. After a milestone, iteration, assignee, or weight is assigned to an issue board, you can no longer -filter through these in the search bar. In order to do that, you need to remove the desired scope +filter through these in the search bar. To do that, you need to remove the desired scope (for example, milestone, assignee, or weight) from the issue board. If you don't have editing permission in a board, you're still able to see the configuration by @@ -255,14 +277,6 @@ selecting **View scope**. Watch a [video presentation](https://youtu.be/m5UTNCSqaDk) of the configurable issue board feature. -### Focus mode - -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28597) to GitLab Free SaaS in 12.10. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212331) to GitLab Free self-managed in 13.0. - -To enable or disable focus mode, select the **Toggle focus mode** button (**{maximize}**) at the top -right. In focus mode, the navigation UI is hidden, allowing you to focus on issues in the board. - ### Sum of issue weights **(PREMIUM)** > Moved to GitLab Premium in 13.9. @@ -273,13 +287,6 @@ especially in combination with [assignee lists](#assignee-lists). ![issue board summed weights](img/issue_board_summed_weights_v13_6.png) -### Group issue boards - -Accessible at the group navigation level, a group issue board offers the same features as a project-level board. -It can display issues from all projects that fall under the group and its descendant subgroups. - -Users on GitLab Free can use a single group issue board. - ### Assignee lists **(PREMIUM)** As in a regular list showing all issues with a chosen label, you can add @@ -394,11 +401,11 @@ You can also [drag issues](#move-issues-and-lists) to change their position and ![Drag issues between swimlanes](img/epics_swimlanes_drag_and_drop.png) -## Work In Progress limits **(PREMIUM)** +## Work in progress limits **(PREMIUM)** > Moved to GitLab Premium in 13.9. -You can set a Work In Progress (WIP) limit for each issue list on an issue board. When a limit is +You can set a work in progress (WIP) limit for each issue list on an issue board. When a limit is set, the list's header shows the number of issues in the list and the soft limit of issues. You cannot set a WIP limit on the default lists (**Open** and **Closed**). @@ -413,11 +420,11 @@ Prerequisites: - You must have at least the Reporter role for the project. -To set a WIP limit for a list: +To set a WIP limit for a list, in an issue board: -1. Navigate to a Project or Group board of which you're a member. -1. Select the settings icon in a list's header. -1. Next to **Work In Progress Limit**, select **Edit**. +1. On the top of the list you want to edit, select **List actions** (**{ellipsis_v}**) **> Edit list settings**. + The list settings sidebar opens on the right. +1. Next to **Work in progress limit**, select **Edit**. 1. Enter the maximum number of issues. 1. Press <kbd>Enter</kbd> to save. @@ -439,7 +446,6 @@ When you hover over the blocked icon (**{issue-block}**), a detailed information - [Remove an existing list](#remove-a-list). - [Remove an issue from a list](#remove-an-issue-from-a-list). - [Filter issues](#filter-issues) that appear across your issue board. -- [Create workflows](#create-workflows). - [Move issues and lists](#move-issues-and-lists). - [Multi-select issue cards](#multi-select-issue-cards). - Drag and reorder the lists. @@ -475,7 +481,7 @@ Additionally, you can also see the time tracking value. ### Create a new list -Create a new list by selecting the **Create** button in the upper right corner of the issue board. +To create a new list, in the upper-right corner of the issue board, select **Create**. ![creating a new list in an issue board](img/issue_board_add_list_v14_1.png) @@ -493,10 +499,10 @@ Prerequisites: To remove a list from an issue board: -1. On the top of the list you want to remove, select the **List settings** icon (**{settings}**). +1. On the top of the list you want to remove, select **List actions** (**{ellipsis_v}**). The list settings sidebar opens on the right. 1. Select **Remove list**. A confirmation dialog appears. -1. Select **OK**. +1. Select **Remove list** again. ### Add issues to a list @@ -567,44 +573,6 @@ When [filtering issues](#filter-issues) in a **group** board, keep this behavior When you edit issues individually using the right sidebar, you can additionally select the milestones and labels from the **project** that the issue is from. -### Create workflows - -By reordering your lists, you can create workflows. As lists in issue boards are -based on labels, it works out of the box with your existing issues. - -So if you've already labeled things with **Backend** and **Frontend**, the issue appears in -the lists as you create them. In addition, this means you can move something between lists by -changing a label. - -A typical workflow of using an issue board would be: - -1. You [create](labels.md#create-a-label) and [prioritize](labels.md#set-label-priority) - labels to categorize your issues. -1. You have a bunch of issues (ideally labeled). -1. You visit the issue board and start [creating lists](#create-a-new-list) to - create a workflow. -1. You move issues around in lists so that your team knows who should be working - on what issue. -1. When the work by one team is done, the issue can be dragged to the next list - so someone else can pick it up. -1. When the issue is finally resolved, the issue is moved to the **Done** list - and gets automatically closed. - -For example, you can create a list based on the label of **Frontend** and one for -**Backend**. A designer can start working on an issue by adding it to the -**Frontend** list. That way, everyone knows that this issue is now being -worked on by the designers. - -Then, when they're done, all they have to do is -drag it to the next list, **Backend**. Then, a backend developer can -eventually pick it up. When they're done, they move it to **Done**, to close the -issue. - -This process can be seen clearly when visiting an issue. With every move -to another list, the label changes and a system note is recorded. - -![issue board system notes](img/issue_board_system_notes_v13_6.png) - ### Move issues and lists You can move issues and lists by dragging them. diff --git a/doc/user/project/issues/associate_zoom_meeting.md b/doc/user/project/issues/associate_zoom_meeting.md index c7187323850..d286e784e0a 100644 --- a/doc/user/project/issues/associate_zoom_meeting.md +++ b/doc/user/project/issues/associate_zoom_meeting.md @@ -36,7 +36,7 @@ You are only allowed to attach a single Zoom meeting to an issue. If you attempt to add a second Zoom meeting using the `/zoom` quick action, it doesn't work. You need to [remove it](#removing-an-existing-zoom-meeting-from-an-issue) first. -Users on GitLab Premium and higher can also +Users on GitLab Premium and Ultimate can also [add multiple Zoom links to incidents](../../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident). ## Removing an existing Zoom meeting from an issue diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 2b302a60d63..79278d1b3ad 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -85,18 +85,21 @@ Learn how to create [merge requests for confidential issues](../merge_requests/c There are two kinds of level access for confidential issues. The general rule is that confidential issues are visible only to members of a project with at -least the Reporter role. However, a guest user can also create +least the **Reporter role**. + +However, a user with the **Guest role** can create confidential issues, but can only view the ones that they created themselves. -Users with the Guest role or non-members can also read the confidential issue if they are assigned to the issue. + +Users with the Guest role or non-members can read the confidential issue if they are assigned to the issue. When a Guest user or non-member is unassigned from a confidential issue, they can no longer view it. -Confidential issues are also hidden in search results for unprivileged users. +Confidential issues are hidden in search results for unprivileged users. For example, here's what a user with the Maintainer role and the Guest role -sees in the project's search results respectively. +sees in the project's search results: -| Maintainer role | Guest role | -|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| +| Maintainer role | Guest role | +|:----------------|:-----------| | ![Confidential issues search by maintainer](img/confidential_issues_search_master.png) | ![Confidential issues search by guest](img/confidential_issues_search_guest.png) | ## Related topics diff --git a/doc/user/project/issues/create_issues.md b/doc/user/project/issues/create_issues.md index 5ebb2fc2e1c..4511c89b0ff 100644 --- a/doc/user/project/issues/create_issues.md +++ b/doc/user/project/issues/create_issues.md @@ -78,7 +78,7 @@ Prerequisites: To create an issue from another issue: -1. In an existing issue, select the vertical ellipsis (**{ellipsis_v}**). +1. In an existing issue, select **Issue actions** (**{ellipsis_v}**). 1. Select **New related issue**. 1. Complete the [fields](#fields-in-the-new-issue-form). The new issue form has a **Relate to issue #123** checkbox, where `123` is the ID of the @@ -100,7 +100,7 @@ To create an issue from a project issue board: 1. On the top bar, select **Main menu > Projects** and find your project. 1. Select **Issues > Boards**. -1. At the top of a board list, select **New issue** (**{plus-square}**). +1. At the top of a board list, select **List actions** (**{ellipsis_v}**) **> Create new issue**. 1. Enter the issue's title. 1. Select **Create issue**. @@ -108,7 +108,7 @@ To create an issue from a group issue board: 1. On the top bar, select **Main menu > Groups** and find your group. 1. Select **Issues > Boards**. -1. At the top of a board list, select **New issue** (**{plus-square}**). +1. At the top of a board list, select **List actions** (**{ellipsis_v}**) **> Create new issue**. 1. Enter the issue's title. 1. Under **Projects**, select the project in the group that the issue should belong to. 1. Select **Create issue**. @@ -118,9 +118,6 @@ example, if the list is scoped to a label `Frontend`, the new issue also has thi ## By sending an email -> - Generated email address format changed in GitLab 11.7. -> - The older format is still supported, so existing aliases and contacts still work. - You can send an email to create an issue in a project on the project's **Issues List** page. diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index 52da1acd32a..f1f41de7cb4 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -39,10 +39,10 @@ git commit -m "this is my commit message. Ref projectname#xxx" ``` If they are not in the same group, you can add the full URL to the issue -(`https://gitlab.com/<username>/<projectname>/issues/<xxx>`). +(`https://gitlab.com/<username>/<projectname>/-/issues/<xxx>`). ```shell -git commit -m "this is my commit message. Related to https://gitlab.com/<username>/<projectname>/issues/<xxx>" +git commit -m "this is my commit message. Related to https://gitlab.com/<username>/<projectname>/-/issues/<xxx>" ``` Of course, you can replace `gitlab.com` with the URL of your own GitLab instance. diff --git a/doc/user/project/issues/csv_import.md b/doc/user/project/issues/csv_import.md index 8a6683a55b5..d23650e973a 100644 --- a/doc/user/project/issues/csv_import.md +++ b/doc/user/project/issues/csv_import.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Import +group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -36,8 +36,8 @@ To import issues: 1. Go to your project's Issues list page. 1. Open the import feature, depending if the project has issues: - - Existing issues are present: Select the import icon in the upper right, next to **Edit issues**. - - Project has no issues: Select **Import CSV** in the middle of the page. + - The project has existing issues: in the upper-right corner, next to **Bulk edit**, select **Actions** (**{ellipsis_v}**) **> Import CSV**. + - The project has no issues: in the middle of the page, select **Import CSV**. 1. Select the file you want to import, and then select **Import issues**. The file is processed in the background, and a notification email is sent diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index f43f87119a6..10b29feb072 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -227,6 +227,19 @@ New discussion threads get different pin numbers, which you can use to refer to In GitLab 12.5 and later, new discussions are output to the issue activity, so that everyone involved can participate in the discussion. +## Delete a comment from a design + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385100) in GitLab 15.9. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To delete a comment from a design: + +1. On the comment you want to delete, select **More actions** **{ellipsis_v}** **> Delete comment**. +1. On the confirmation dialog box, select **Delete comment**. + ## Resolve a discussion thread on a design > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13049) in GitLab 13.1. diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index f41c90a74a5..63b60e4538f 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -49,7 +49,8 @@ server's time zone. Issues with due dates can also be exported as an iCalendar feed. The URL of the feed can be added to calendar applications. The feed is accessible by selecting -the **Subscribe to calendar** button on the following pages: +the **Subscribe to calendar** option in the **Actions** (**{ellipsis_v}**) dropdown +list on the following pages: - The **Assigned Issues** page linked on the right side of the GitLab header - The **Project Issues** page diff --git a/doc/user/project/issues/issue_weight.md b/doc/user/project/issues/issue_weight.md index d852ad3262b..05c348acf58 100644 --- a/doc/user/project/issues/issue_weight.md +++ b/doc/user/project/issues/issue_weight.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/issue_weight.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index c16074ea1d8..276cc3bc7a5 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -54,7 +54,7 @@ To edit multiple issues at the same time: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Issues**. -1. Select **Edit issues**. A sidebar on the right of your screen appears. +1. Select **Bulk edit**. A sidebar on the right of your screen appears. 1. Select the checkboxes next to each issue you want to edit. 1. From the sidebar, edit the available fields. 1. Select **Update all**. @@ -87,7 +87,7 @@ To edit multiple issues at the same time: 1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Issues**. -1. Select **Edit issues**. A sidebar on the right of your screen appears. +1. Select **Bulk edit**. A sidebar on the right of your screen appears. 1. Select the checkboxes next to each issue you want to edit. 1. From the sidebar, edit the available fields. 1. Select **Update all**. @@ -103,7 +103,7 @@ When bulk editing issues in a group, you can edit the following attributes: ## Move an issue When you move an issue, it's closed and copied to the target project. -The original issue is not deleted. A system note, which indicates +The original issue is not deleted. A [system note](../system_notes.md), which indicates where it came from and went to, is added to both issues. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. @@ -136,7 +136,7 @@ To move multiple issues at the same time: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Issues**. -1. Select **Edit issues**. A sidebar on the right of your screen appears. +1. Select **Bulk edit**. A sidebar on the right of your screen appears. 1. Select the checkboxes next to each issue you want to move. 1. From the right sidebar, select **Move selected**. 1. From the dropdown list, select the destination project. @@ -209,6 +209,10 @@ To close an issue, you can do the following: - At the top of the issue, select **Close issue**. - In an [issue board](../issue_board.md), drag an issue card from its list into the **Closed** list. +<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> +If you don't see this action at the top of an issue, your project or instance might have +enabled a feature flag for [moved actions](../merge_requests/index.md#move-sidebar-actions). + ### Reopen a closed issue Prerequisites: @@ -298,7 +302,7 @@ To disable automatic issue closing: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. -1. Expand **Default branch**. +1. Expand **Branch defaults**. 1. Clear the **Auto-close referenced issues on default branch** checkbox. 1. Select **Save changes**. @@ -344,7 +348,7 @@ Prerequisites: To delete an issue: -1. In an issue, select the vertical ellipsis (**{ellipsis_v}**). +1. In an issue, select **Issue actions** (**{ellipsis_v}**). 1. Select **Delete issue**. Alternatively: @@ -362,7 +366,7 @@ You can promote an issue to an [epic](../../group/epics/index.md) in the immedia To promote an issue to an epic: -1. In an issue, select the vertical ellipsis (**{ellipsis_v}**). +1. In an issue, select **Issue actions** (**{ellipsis_v}**). 1. Select **Promote to epic**. Alternatively, you can use the `/promote` [quick action](../quick_actions.md#issues-merge-requests-and-epics). @@ -400,7 +404,7 @@ To view all issues assigned to you: Or: - To use a [keyboard shortcut](../../shortcuts.md), press <kbd>Shift</kbd> + <kbd>i</kbd>. -- On the top bar, in the upper right, select **{issues}** **Issues**. +- On the top bar, in the upper-right corner, select **Issues** (**{issues}**). ## Filter the list of issues @@ -472,6 +476,10 @@ You can now paste the reference into another description or comment. Read more about issue references in [GitLab-Flavored Markdown](../../markdown.md#gitlab-specific-references). +<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> +If you don't see this action on the right sidebar, your project or instance might have +enabled a feature flag for [moved actions](../merge_requests/index.md#move-sidebar-actions). + ## Copy issue email address > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18816) in GitLab 13.8. diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index a2f90d5c444..829e44eae9a 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -73,7 +73,7 @@ then issues with a milestone without a due date. ## Sorting by popularity When you sort by **Popularity**, the issue order changes to sort descending by the -number of upvotes ([awarded](../../award_emojis.md) a "thumbs up" emoji) +number of upvotes ([emoji reactions](../../award_emojis.md) with the "thumbs up") on each issue. You can use this to identify issues that are in high demand. ## Sorting by priority diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index f9e3f1b9225..2af15e06b98 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -136,7 +136,7 @@ To do so: 1. Fill in the name field. You can't specify a description if creating a label this way. You can add a description later by [editing the label](#edit-a-label). 1. Select a color by selecting from the available colors, or enter a hex color value for a specific color. -1. Select **Create**. +1. Select **Create**. Your label is created and selected. ### Create a group label @@ -381,7 +381,7 @@ issue is now under review, they assign the `workflow::review`, and the `workflow is removed. The same happens when you move issues across label lists in an -[issue board](issue_board.md#create-workflows). With scoped labels, team members not working in an +[issue board](issue_board.md). With scoped labels, team members not working in an issue board can also advance workflow states consistently in issues themselves. For a video explanation, see: diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index c7d45b0bd15..6e20492db05 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 4dda68a6d08..452363c3d9a 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -28,17 +28,21 @@ To invite a group to a project, you must be at least one of the following: In addition: -- The group you're inviting must have a more restrictive - [visibility level](../../public_access.md#project-and-group-visibility) - than the project. For example, you can invite: - - A private group to a public project. - - An internal group to a public project. - - A private group to an internal project. +- You must be a member of the group or the subgroup being invited. -- The group or subgroup must be in the project's [namespace](../../namespace/index.md). +- The [visibility level](../../public_access.md#project-and-group-visibility) of the group you're inviting +must be at least as restrictive as that of the project. For example, you can invite: + - A _private_ group to a _private_ project + - A _private_ group to an _internal_ project. + - A _private_ group to a _public_ project. + - An _internal_ group to an _internal_ project. + - An _internal_ group to a _public_ project. + - A _public_ group to a _public_ project. + +- If the project's root ancestor group [does not allow the project to be shared outside the hierarchy](../../group/access_and_permissions.md#prevent-group-sharing-outside-the-group-hierarchy), the invited group or subgroup must be in the project's [namespace](../../namespace/index.md). For example, a project in the namespace `group/subgroup01/project`: - Can be shared with `group/subgroup02` or `group/subgroup01/subgroup03`. - - Cannot be shared with `group`. + - Cannot be shared with `group_abc`. ## Share a project with a group @@ -70,17 +74,21 @@ are given access to the project. In addition: ## Maximum role +When you invite a group to a project, the maximum role is the highest level of access the invited group members are allowed to have in the project. + When multiple groups contain the same members, and the groups have access to the same project, the group members are -given the most restrictive role for the project. - -This most restrictive role is called the *maximum role*, or **Max role**. +given the highest access level of the two for the project. The member's **Max role** is the more restrictive of: - The role the user is assigned for the group. - The role you chose when you invited the group to the project. +NOTE: +The Max role does not elevate the privileges of users. +For example, if a group member has the role of Developer, and the group is invited to a project with a Max role of Maintainer, the member's role is not elevated to Maintainer. + ### View the member's Max role To view the maximum role assigned to a member: diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md index 0729e024df4..d5851050fd4 100644 --- a/doc/user/project/merge_requests/allow_collaboration.md +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -61,14 +61,14 @@ In the following example: To change or add a commit to the contributor's merge request: 1. Go to the merge request. -1. In the upper right corner, select **Code**, then select **Check out branch**. +1. In the upper-right corner, select **Code**, then select **Check out branch**. 1. In the modal window, select **Copy** (**{copy-to-clipboard}**). 1. In your terminal, go to your cloned version of the repository, and paste the commands. For example: ```shell git fetch "git@gitlab.com:contributor/forked-project.git" 'fork-branch' - git checkout -b contributor/fork-branch' FETCH_HEAD + git checkout -b 'contributor/fork-branch' FETCH_HEAD ``` Those commands fetch the branch from the forked project, and create a local branch diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index 21e2055cb61..717358df9f3 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -3,7 +3,6 @@ stage: Create group: Source Code info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/approvals/index.html' --- # Merge request approvals **(FREE)** @@ -17,7 +16,7 @@ approve merge requests, these approvals are [optional](#optional-approvals). flexibility: - Create required [rules](rules.md) about the number and type of approvers before work can merge. -- Specify a list of users who act as [code owners](../../code_owners.md) for specific files, +- Specify a list of users who act as [code owners](../../codeowners/index.md) for specific files, and require their approval before work can merge. You can configure merge request approvals on a per-project basis, and some approvals can be configured @@ -36,7 +35,7 @@ required approvals before work can merge into your project. You can also extend rules to define what types of users can approve work. Some examples of rules you can create include: - Users with specific permissions can always approve work. -- [Code owners](../../code_owners.md) can approve work for files they own. +- [Code owners](../../codeowners/index.md) can approve work for files they own. - Users with specific permissions can approve work, [even if they don't have merge rights](rules.md#merge-request-approval-segregation-of-duties) to the repository. @@ -102,20 +101,31 @@ Without the approvals, the work cannot merge. Required approvals enable multiple database, for all proposed code changes. - Use the [code owners of changed files](rules.md#code-owners-as-eligible-approvers), to determine who should review the work. -- Require an [approval before merging code that causes test coverage to decline](../../../../ci/pipelines/settings.md#coverage-check-approval-rule) +- Require an [approval before merging code that causes test coverage to decline](../../../../ci/testing/code_coverage.md#coverage-check-approval-rule). - Users on GitLab Ultimate can also [require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) before merging code that could introduce a vulnerability. ## Invalid rules -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334698) in GitLab 15.1. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334698) in GitLab 15.1. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/389905) in GitLab 15.11 [with a flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. Disabled by default. -Whenever an approval rule cannot be satisfied, the rule will be displayed as `Invalid`. This applies to the following conditions: +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, +ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. +On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. + +Whenever an approval rule cannot be satisfied, the rule is displayed as **(!) Auto approved**. This applies to the following conditions: - The only eligible approver is the author of the merge request. - No eligible approvers (either groups or users) have been assigned to the approval rule. +- The number of required approvals is more than the number of eligible approvers. + +These rules are automatically approved to unblock their respective merge requests, +unless they were created through a security policy. -These rules will be automatically approved to unblock their respective merge requests. +Invalid approval rules created through a security policy are presented with **(!) Action Required** +and are not automatically approved, blocking their respective merge requests. ## Related topics diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index 0a5e7c29860..d55ca5dff04 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -53,7 +53,7 @@ To add a merge request approval rule: 1. Select **Add approval rule**. -Users of GitLab Premium and higher tiers can create [additional approval rules](#add-multiple-approval-rules). +Users of GitLab Premium and Ultimate tiers can create [additional approval rules](#add-multiple-approval-rules). Your configuration for approval rule overrides determines if the new rule is applied to existing merge requests: @@ -91,7 +91,7 @@ To edit a merge request approval rule: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1979) in GitLab 11.10. -In GitLab Premium and higher tiers, you can enforce multiple approval rules on a +In GitLab Premium and Ultimate tiers, you can enforce multiple approval rules on a merge request, and multiple default approval rules for a project. If your tier supports multiple default rules: @@ -106,6 +106,9 @@ reduces the number of approvals left (the **Approvals** column) for all rules th ![Approvals premium merge request widget](img/approvals_premium_mr_widget_v13_3.png) +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Multiple Approvers](https://www.youtube.com/watch?v=8JQJ5821FrA). + ## Eligible approvers > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10294) in GitLab 13.3, when an eligible approver comments on a merge request, it appears in the **Commented by** column of the Approvals widget. @@ -131,7 +134,7 @@ who commented on the merge request. It helps authors and reviewers identify who contact with questions about the merge request's content. If the number of required approvals is greater than the number of assigned approvers, -approvals from other users with Developer [permissions](../../../permissions.md) or higher +approvals from other users with at least the Developer [role](../../../permissions.md) in the project counts toward meeting the required number of approvals, even if the users were not explicitly listed in the approval rules. @@ -158,7 +161,7 @@ approve in these ways: > Moved to GitLab Premium in 13.9. -If you add [code owners](../../code_owners.md) to your repository, the owners of files +If you add [code owners](../../codeowners/index.md) to your repository, the owners of files become eligible approvers in the project. To enable this merge request approval rule: 1. Go to your project and select **Settings > Merge requests**. @@ -241,6 +244,28 @@ approval rule for certain branches: 1. To enable this configuration, read [Code Owner's approvals for protected branches](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch). +## Code coverage check approval + +You can require specific approvals in the case that a merge request would reduce code test +coverage. + +For more information, see [Coverage check approval rule](../../../../ci/testing/code_coverage.md#coverage-check-approval-rule). + +## Security Approvals **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. + +You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. +Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. + +The security approval rules are applied to all merge requests until the pipeline is complete. The application of the +security approval rules prevents users from merging in code before the security scans run. After the pipeline is +complete, the security approval rules are checked to determine if the security approvals are still required. + +![Security Approvals](img/security_approvals_v15_0.png) + +These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). + ## Troubleshooting ### Approval rule name can't be blank @@ -262,18 +287,3 @@ isn't recognized as a valid Code Owner for the project, nor does it display in t project's **Approvals** list. To fix this problem, add the approval group as a shared group high enough in the shared hierarchy so the project requiring review inherits this group of users. - -## Security Approvals **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. - -You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. -Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. - -The security approval rules are applied to all merge requests until the pipeline is complete. The application of the -security approval rules prevents users from merging in code before the security scans run. Once the pipeline is -complete, the security approval rules are checked to determine if the security approvals are still required. - -![Security Approvals](img/security_approvals_v15_0.png) - -These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 1ab564c108b..6c079dc9319 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -78,13 +78,13 @@ their own. To do this: it can't be changed at the project level. 1. Select **Save changes**. -Depending on your version of GitLab, [code owners](../../code_owners.md) who commit +Depending on your version of GitLab, [code owners](../../codeowners/index.md) who commit to a merge request may or may not be able to approve the work: -- In GitLab 13.10 and earlier, [code owners](../../code_owners.md) who commit +- In GitLab 13.10 and earlier, code owners who commit to a merge request can approve it, even if the merge request affects files they own. - In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/331548), - [code owners](../../code_owners.md) who commit + code owners who commit to a merge request cannot approve it, when the merge request affects files they own. For more information, see the [official Git documentation](https://git-scm.com/book/en/v2/Git-Basics-Viewing-the-Commit-History). @@ -121,7 +121,7 @@ permission enables an electronic signature for approvals, such as the one define ## Remove all approvals when commits are added to the source branch By default, an approval on a merge request is removed when you add more changes -after the approval. In GitLab Premium and higher tiers, to keep existing approvals +after the approval. In GitLab Premium and Ultimate tiers, to keep existing approvals after more changes are added to the merge request: 1. On the left sidebar, select **Settings > Merge requests**. @@ -140,7 +140,7 @@ However, approvals are reset if the target branch is changed. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90578) in GitLab 15.3. -If you only want to remove approvals by Code Owners whose files have been changed: +If you only want to remove approvals by Code Owners whose files have been changed when a commit is added: Prerequisite: @@ -153,13 +153,6 @@ To do this: select **Remove approvals by Code Owners if their files changed**. 1. Select **Save changes**. -## Code coverage check approvals - -You can require specific approvals if a merge request would result in a decline in code test -coverage. - -For more information, see [Coverage check approval rule](../../../../ci/pipelines/settings.md#coverage-check-approval-rule). - ## Settings cascading > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) in GitLab 14.4. [Deployed behind the `group_merge_request_approval_settings_feature_flag` flag](../../../../administration/feature_flags.md), disabled by default. diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index 9fac872ac4b..004d24778b7 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -55,7 +55,7 @@ by the merge request: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**, and find your merge request. 1. Scroll to the merge request reports section, and find the **Merged by** report. -1. In the upper right, select **Cherry-pick**: +1. In the upper-right corner, select **Cherry-pick**: ![Cherry-pick merge request](img/cherry_pick_v15_4.png) 1. In the modal window, select the project and branch to cherry-pick into. @@ -72,7 +72,8 @@ To cherry-pick a commit from the list of all commits for a project: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Commits**. -1. Select the title of the commit you want to cherry-pick. +1. Select the [title](https://git-scm.com/docs/git-commit#_discussion) of the commit you want to cherry-pick. +1. In the upper-right corner, select **Options > Cherry-pick** to show the cherry-pick modal. 1. In the modal window, select the project and branch to cherry-pick into. 1. Optional. Select **Start a new merge request with these changes**. 1. Select **Cherry-pick**. @@ -86,7 +87,7 @@ list of commits included in a merge request: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**, and find your merge request. 1. In the merge request's secondary menu, select **Commits** to display the commit details page. -1. Select the title of the commit you want to cherry-pick. +1. Select the [title](https://git-scm.com/docs/git-commit#_discussion) of the commit you want to cherry-pick. 1. In the upper-right corner, select **Options > Cherry-pick** to show the cherry-pick modal. 1. In the modal window, select the project and branch to cherry-pick into. 1. Optional. Select **Start a new merge request with these changes**. @@ -100,7 +101,8 @@ when you view that file in your project's Git repository: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Repository > Files** and go to the file changed by the commit. -1. Select **History**, then select the title of the commit you want to cherry-pick. +1. Select **History**, then select the [title](https://git-scm.com/docs/git-commit#_discussion) + of the commit you want to cherry-pick. 1. In the upper-right corner, select **Options > Cherry-pick** to show the cherry-pick modal. 1. In the modal window, select the project and branch to cherry-pick into. 1. Optional. Select **Start a new merge request with these changes**. @@ -123,7 +125,7 @@ project, from the GitLab user interface: ## View system notes for cherry-picked commits -When you cherry-pick a merge commit in the GitLab UI or API, GitLab adds a system note +When you cherry-pick a merge commit in the GitLab UI or API, GitLab adds a [system note](../system_notes.md) to the related merge request thread in the format **{cherry-pick-commit}** `[USER]` **picked the changes into the branch** `[BRANCHNAME]` with commit** `[SHA]` `[DATE]`: diff --git a/doc/user/project/merge_requests/commits.md b/doc/user/project/merge_requests/commits.md index a9f67c39ae8..cc6ecd8398f 100644 --- a/doc/user/project/merge_requests/commits.md +++ b/doc/user/project/merge_requests/commits.md @@ -1,11 +1,59 @@ --- -redirect_to: '../merge_requests/index.md' -remove_date: '2023-03-12' +stage: Create +group: Code Review +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference --- -This document was removed. +# Merge request commits **(FREE)** -<!-- This redirect file can be deleted after <2023-03-12>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> +Each merge request has a history of the commits made to the source branch +after the merge request was created. + +These commits are displayed on the merge request's **Commits** tab. +From this tab, you can review commit messages and copy a commit's SHA when you need to +[cherry-pick changes](cherry_pick_changes.md). + +## Navigate merge request commits + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18140) in GitLab 13.0. + +To navigate commits in a merge request: + +1. Select the **Commits** tab. +1. Select the commit link. The most recent commit is displayed. +1. Navigate through the commits by either: + + - Selecting **Prev** and **Next** buttons below the tab buttons. + - Using the <kbd>X</kbd> and <kbd>C</kbd> keyboard shortcuts. + +![Merge requests commit navigation](img/commit_nav_v16_0.png) + +## View merge request commits in context + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29274) in GitLab 13.12 [with a flag](../../../administration/feature_flags.md) named `context_commits`. Enabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) in GitLab 14.8. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) in GitLab 14.9. [Feature flag `context_commits`](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) removed. + +When reviewing a merge request, it helps to have more context about the changes +made. That includes unchanged lines in unchanged files, and previous commits +that have already merged that the change is built on. + +To add previously merged commits to a merge request for more context: + +1. Go to your merge request. +1. Select the **Commits** tab. +1. Scroll to the end of the list of commits, and select **Add previously merged commits**: +1. Select the commits that you want to add. +1. Select **Save changes**. + +## View diffs between commits + +To view the changes between previously merged commits: + +1. On your merge request, select the **Changes** tab. +1. By **Compare**, select the commit you want to view: + + ![Previously merged commits](img/previously_merged_commits_v16_0.png) + +If you selected to add previously merged commits, they are displayed in the list. diff --git a/doc/user/project/merge_requests/confidential.md b/doc/user/project/merge_requests/confidential.md index 307998f697d..edcc5711b98 100644 --- a/doc/user/project/merge_requests/confidential.md +++ b/doc/user/project/merge_requests/confidential.md @@ -65,7 +65,7 @@ Your branch, merge requests, and commits remain in your private fork. This preve prematurely revealing confidential information. Open a merge request -[from your fork to the upstream repository](../repository/forking_workflow.md#merging-upstream) when: +[from your fork to the upstream repository](../repository/forking_workflow.md#merge-changes-back-upstream) when: - You are satisfied the problem is resolved in your private fork. - You are ready to make the confidential commits public. diff --git a/doc/user/project/merge_requests/conflicts.md b/doc/user/project/merge_requests/conflicts.md index 932eec5e631..cc8f8cb2fe6 100644 --- a/doc/user/project/merge_requests/conflicts.md +++ b/doc/user/project/merge_requests/conflicts.md @@ -162,9 +162,8 @@ merge commit. Verify it contains no unintended changes and doesn't break your bu ## Related topics -- [Introduction to Git rebase and force-push](../../../topics/git/git_rebase.md). -- [Git GUI apps](https://git-scm.com/downloads/guis) to help you visualize the - differences between branches and resolve them. +- [Introduction to Git rebase and force-push](../../../topics/git/git_rebase.md) +- [Git applications for visualizing the Git workflow](https://git-scm.com/downloads/guis) <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index 875312bbb7c..a91d324016a 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -3,15 +3,17 @@ stage: Create group: Code Review info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: "How to create merge requests in GitLab." -disqus_identifier: 'https://docs.gitlab.com/ee/gitlab-basics/add-merge-request.html' --- # Creating merge requests **(FREE)** -There are many different ways to create a merge request. +GitLab provides many different ways to create a merge request. NOTE: -Use [branch naming patterns](../repository/branches/index.md#naming) to streamline merge request creation. +GitLab enforces [branch naming rules](../repository/branches/index.md#name-your-branch) +to prevent problems, and provides +[branch naming patterns](../repository/branches/index.md#prefix-branch-names-with-issue-numbers) +to streamline merge request creation. ## From the merge request list @@ -19,7 +21,7 @@ You can create a merge request from the list of merge requests. 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Merge requests**. -1. In the upper right, select **New merge request**. +1. In the upper-right corner, select **New merge request**. 1. Select a source and target branch and then **Compare branches and continue**. 1. Fill out the fields and select **Create merge request**. @@ -34,7 +36,8 @@ be associated with a given target branch at a time. If your development workflow requires an issue for every merge request, you can create a branch directly from the issue to speed the process up. The new branch, and later its merge request, are marked as related to this issue. -After merging the merge request, the issue is closed automatically, unless [automatic issue closing is disabled](../issues/managing_issues.md#disable-automatic-issue-closing). +After merging the merge request, the issue is closed automatically, unless +[automatic issue closing is disabled](../issues/managing_issues.md#disable-automatic-issue-closing). You can see a **Create merge request** dropdown list below the issue description. NOTE: @@ -50,7 +53,7 @@ instead of immediately creating the merge request. - Your project is private and the issue is confidential. To make this button appear, one possible workaround is to -[remove your project's fork relationship](../settings/index.md#remove-a-fork-relationship). +[remove your project's fork relationship](../repository/forking_workflow.md#unlink-a-fork). After removal, the fork relationship cannot be restored. This project can no longer be able to receive or send merge requests to the source project, or other forks. @@ -58,8 +61,8 @@ The dropdown list contains the options **Create merge request and branch** and * After selecting one of these options, a new branch or branch and merge request is created based on your project's [default branch](../repository/branches/default.md). -The branch name is based on your project's branch name template. The default template -is `%{id}-%{title}`. Supported variables for branch name templates are `%{id}` and `%{title}`. +The branch name is based on your project's [branch name template](../repository/branches/index.md), +but this value can be changed. When you select **Create branch** in an empty repository project, GitLab performs these actions: @@ -83,7 +86,7 @@ You can create a merge request when you add, edit, or upload a file to a reposit 1. [Add, edit, or upload](../repository/web_editor.md) a file to the repository. 1. In the **Commit message**, enter a reason for the commit. -1. Select the **Target branch** or create a new branch by typing the name (without spaces, capital letters, or special chars). +1. Select the **Target branch** or create a new branch by typing the name (without spaces). 1. Select the **Start a new merge request with these changes** checkbox or toggle. This checkbox or toggle is visible only if the target is not the same as the source branch, or if the source branch is protected. 1. Select **Commit changes**. @@ -151,9 +154,8 @@ You can create a merge request from your fork to contribute back to the main pro 1. Select **Create merge request**. After your work is merged, if you don't intend to -make any other contributions to the upstream project, you can unlink your -fork from its upstream project. Go to **Settings > Advanced Settings** and -[remove the forking relationship](../settings/index.md#remove-a-fork-relationship). +make any other contributions to the upstream project, you can +[unlink your fork](../repository/forking_workflow.md#unlink-a-fork) from its upstream project. For more information, [see the forking workflow documentation](../repository/forking_workflow.md). @@ -171,7 +173,7 @@ To create a merge request by sending an email: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left menu, select **Merge requests**. -1. In the upper right, select **Email a new merge request to this project**. +1. In the upper-right corner, select **Email a new merge request to this project**. An email address is displayed. Copy this address. Ensure you keep this address private. 1. Open an email and compose a message with the following information: diff --git a/doc/user/project/merge_requests/csv_export.md b/doc/user/project/merge_requests/csv_export.md index 9028a9411ea..f40b82a6280 100644 --- a/doc/user/project/merge_requests/csv_export.md +++ b/doc/user/project/merge_requests/csv_export.md @@ -16,7 +16,7 @@ To export merge requests to a CSV file: 1. On the left sidebar, select **Merge requests** . 1. Add any searches or filters. This can help you keep the size of the CSV file under the 15 MB limit. The limit ensures the file can be emailed to a variety of email providers. -1. Select **Export as CSV** (**{export}**). +1. Select **Actions** (**{ellipsis_v}**) **> Export as CSV**. 1. Confirm the correct number of merge requests are to be exported. 1. Select **Export merge requests**. diff --git a/doc/user/project/merge_requests/dependencies.md b/doc/user/project/merge_requests/dependencies.md index 231c1dda5b7..3d92fc9a91e 100644 --- a/doc/user/project/merge_requests/dependencies.md +++ b/doc/user/project/merge_requests/dependencies.md @@ -85,7 +85,7 @@ other specific work merges, even if the merge request is in a different project. Prerequisites: - You must have at least the Developer role or be allowed to create merge requests in the project. -- The dependent merge request must be in a project in a **PREMIUM** or higher tier. +- The dependent merge request must be in a project in the Premium or Ultimate tier. To create a new merge request and mark it as dependent on another: diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 62820988701..88e5e4a6283 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -3,7 +3,6 @@ stage: Create group: Code Review info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/work_in_progress_merge_requests.html' --- # Draft merge requests **(FREE)** @@ -31,7 +30,7 @@ There are several ways to flag a merge request as a draft: below the **Title** field. - **Commenting in an existing merge request**: Add the `/draft` [quick action](../quick_actions.md#issues-merge-requests-and-epics) - in a comment. GitLab 15.4 [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) the toggle behavior of `/draft`. To mark a merge request as ready, use `/ready`. + in a comment. To mark a merge request as ready, use `/ready`. - **Creating a commit**: Add `draft:`, `Draft:`, `fixup!`, or `Fixup!` to the beginning of a commit message targeting the merge request's source branch. This is not a toggle, and adding this text again in a later commit doesn't mark the diff --git a/doc/user/project/merge_requests/img/commit_nav_v16_0.png b/doc/user/project/merge_requests/img/commit_nav_v16_0.png Binary files differnew file mode 100644 index 00000000000..6005e516fff --- /dev/null +++ b/doc/user/project/merge_requests/img/commit_nav_v16_0.png diff --git a/doc/user/project/merge_requests/img/previously_merged_commits_v16_0.png b/doc/user/project/merge_requests/img/previously_merged_commits_v16_0.png Binary files differnew file mode 100644 index 00000000000..31204b6e801 --- /dev/null +++ b/doc/user/project/merge_requests/img/previously_merged_commits_v16_0.png diff --git a/doc/user/project/merge_requests/img/remove_source_branch_status.png b/doc/user/project/merge_requests/img/remove_source_branch_status.png Binary files differdeleted file mode 100644 index afd93207e02..00000000000 --- a/doc/user/project/merge_requests/img/remove_source_branch_status.png +++ /dev/null diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index a633366cc62..a65c5518658 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -26,11 +26,27 @@ view [this GitLab Flow video](https://www.youtube.com/watch?v=InKNIvky2KE). Learn the various ways to [create a merge request](creating_merge_requests.md). +### Use merge request templates + +When you create a merge request, GitLab checks for the existence of a +[description template](../description_templates.md) to add data to your merge request. +GitLab checks these locations in order from 1 to 5, and applies the first template +found to your merge request: + +| Name | Project UI<br>setting | Group<br>`default.md` | Instance<br>`default.md` | Project<br>`default.md` | No template | +| :-- | :--: | :--: | :--: | :--: | :--: | +| Standard commit message | 1 | 2 | 3 | 4 | 5 | +| Commit message with an [issue closing pattern](../issues/managing_issues.md#closing-issues-automatically) like `Closes #1234` | 1 | 2 | 3 | 4 | 5 \* | +| Branch name [prefixed with an issue ID](../repository/branches/index.md#prefix-branch-names-with-issue-numbers), like `1234-example` | 1 \* | 2 \* | 3 \* | 4 \* | 5 \* | + +NOTE: +Items marked with an asterisk (\*) also append an [issue closing pattern](../issues/managing_issues.md#closing-issues-automatically). + ## View merge requests You can view merge requests for your project, group, or yourself. -### View merge requests for a project +### For a project To view all merge requests for a project: @@ -39,7 +55,7 @@ To view all merge requests for a project: Or, to use a [keyboard shortcut](../../shortcuts.md), press <kbd>g</kbd> + <kbd>m</kbd>. -### View merge requests for all projects in a group +### For all projects in a group To view merge requests for all projects in a group: @@ -48,7 +64,7 @@ To view merge requests for all projects in a group: If your group contains subgroups, this view also displays merge requests from the subgroup projects. -### View all merge requests assigned to you +### Assigned to you To view all merge requests assigned to you: @@ -65,7 +81,7 @@ or: or: -1. On the top bar, in the upper right, select **{merge-request-open}** **Merge requests**. +1. On the top bar, in the upper-right corner, select **Merge requests** (**{merge-request-open}**). 1. From the dropdown list, select **Assigned to you**. ## Filter the list of merge requests @@ -79,12 +95,12 @@ To filter the list of merge requests: 1. Above the list of merge requests, select **Search or filter results...**. 1. From the dropdown list, select the attribute you wish to filter by. Some examples: - - [**By environment or deployment date**](#filter-merge-requests-by-environment-or-deployment-date). + - [**By environment or deployment date**](#by-environment-or-deployment-date). - **ID**: Enter filter `#30` to return only merge request 30. - User filters: Type (or select from the dropdown list) any of these filters to display a list of users: - **Approved-By**, for merge requests already approved by a user. **(PREMIUM)**. - **Approver**, for merge requests that this user is eligible to approve. - (For more information, read about [Code owners](../code_owners.md)). **(PREMIUM)** + (For more information, read about [Code owners](../codeowners/index.md)). **(PREMIUM)** - **Reviewer**, for merge requests reviewed by this user. 1. Select or type the operator to use for filtering the attribute. The following operators are available: @@ -100,7 +116,7 @@ To filter the list of merge requests: GitLab displays the results on-screen, but you can also [retrieve them as an RSS feed](../../search/index.md#retrieve-search-results-as-feed). -### Filter merge requests by environment or deployment date +### By environment or deployment date > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44041) in GitLab 13.6. @@ -251,14 +267,15 @@ after merging does not retarget open merge requests. This improvement is <!-- When the `moved_mr_sidebar` feature flag is removed, delete this topic and update the steps for these actions like in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87727/diffs?diff_id=522279685#5d9afba799c4af9920dab533571d7abb8b9e9163 --> -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85584) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85584) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. Disabled by default. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/373757) to also move actions on issues, incidents, and epics in GitLab 16.0. FLAG: On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. On GitLab.com, this feature is enabled in the following projects: `gitlab-org/gitlab`, `gitlab-com/www-gitlab-com`, and `gitlab-org/customers-gitlab-com`. -When this feature flag is enabled, you can find the following actions in -**Merge request actions** (**{ellipsis_v}**) in the upper right: +When this feature flag is enabled, in the upper-right corner, +**Merge request actions** (**{ellipsis_v}**) contains the following actions: - The [notifications](../../profile/notifications.md#edit-notification-settings-for-issues-merge-requests-and-epics) toggle - Mark merge request as ready or [draft](../merge_requests/drafts.md) @@ -266,6 +283,8 @@ When this feature flag is enabled, you can find the following actions in - [Lock discussion](../../discussions/index.md#prevent-comments-by-locking-the-discussion) - Copy reference +In GitLab 16.0 and later, similar action menus are available on issues, incidents, and epics. + When this feature flag is disabled, these actions are in the right sidebar. ## Merge request workflows @@ -295,6 +314,43 @@ For a web developer writing a webpage for your company's website: 1. Once approved, your merge request is [squashed and merged](squash_and_merge.md), and [deployed to staging with GitLab Pages](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/). 1. Your production team [cherry-picks](cherry_pick_changes.md) the merge commit into production. +## Filter activity in a merge request + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115383) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `mr_activity_filters`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. +To make it available per user, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `mr_activity_filters` for individual or groups of users. +On GitLab.com, this feature unavailable. + +To understand the history of a merge request, filter its activity feed to show you +only the items that are relevant to you. + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests**. +1. Select a merge request. +1. Scroll to **Activity**. +1. On the right side of the page, select **Activity filter** to show the filter options. + If you've selected filter options previously, this field shows a summary of your + choices, like **Activity + 5 more**. +1. Select the types of activity you want to see. Options include: + + - Assignees & Reviewers + - Approvals + - Comments + - Commits & branches + - Edits + - Labels + - Lock status + - Mentions + - Merge request status + - Tracking + +1. Optional. Select **Sort** (**{sort-lowest}**) to reverse the sort order. + +Your selection persists across all merge requests. You can also change the +sort order by clicking the sort button on the right. + ## Related topics - [Create a merge request](creating_merge_requests.md) @@ -304,7 +360,6 @@ For a web developer writing a webpage for your company's website: - [GitLab keyboard shortcuts](../../shortcuts.md) - [Comments and threads](../../discussions/index.md) - [Suggest code changes](reviews/suggestions.md) -- [Commits](commits.md) - [CI/CD pipelines](../../../ci/index.md) - [Push options](../push_options.md) for merge requests diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index f0359446b06..7588af70bd4 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -5,7 +5,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, concepts --- -# Merge when pipeline succeeds **(FREE)** +# Auto-merge **(FREE)** + +> **Merge when pipeline succeeds** and **Add to merge train when pipeline succeeds** [renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409530) to **Auto-merge** in GitLab 16.0 [with a flag](../../../administration/feature_flags.md) named `auto_merge_labels_mr_widget`. Enabled by default. + +NOTE: +[In GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/359057), **Merge when pipeline succeeds** and **Add to merge train when pipeline succeeds** become **Set to auto-merge**. If you review a merge request and it's ready to merge, but the pipeline hasn't completed yet, you can set it to merge when the pipeline succeeds (MWPS). You don't @@ -147,3 +152,11 @@ merge-request-pipeline-job: Instead, use branch (`push`) pipelines or merge request pipelines, when possible. For details on avoiding two pipelines for a single merge request, read the [`rules` documentation](../../../ci/jobs/job_control.md#avoid-duplicate-pipelines). + +### Merged results pipeline allows merge, despite a failed branch pipeline + +When [the **Pipelines must succeed** setting](#require-a-successful-pipeline-for-merge) +is combined with +[the **Merged results pipelines** feature](../../../ci/pipelines/merged_results_pipelines.md), +failed branch pipeline may be ignored. +[Issue 385841](https://gitlab.com/gitlab-org/gitlab/-/issues/385841) is open to track this. diff --git a/doc/user/project/merge_requests/methods/index.md b/doc/user/project/merge_requests/methods/index.md index 1f7e15ee982..02bd4ed0502 100644 --- a/doc/user/project/merge_requests/methods/index.md +++ b/doc/user/project/merge_requests/methods/index.md @@ -228,5 +228,4 @@ workflow that requires frequent rebases. ## Related topics -- [Commits history](../commits.md) - [Squash and merge](../squash_and_merge.md) diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 5878873f209..77fd78ee0d0 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -58,7 +58,8 @@ To do this: 1. On the top bar, select **Main menu > Projects** and find your project. 1. If you know the merge request that contains the commit: 1. On the left sidebar, select **Merge requests** and identify your merge request. - 1. Select **Commits**, then select the title of the commit you want to revert. GitLab displays the contents of the commit. + 1. Select **Commits**, then select the title of the commit you want to revert. This displays the commit in the **Changes** tab of your merge request. + 1. Select the commit hash you want to revert. GitLab displays the contents of the commit. 1. If you don't know the merge request the commit originated from: 1. On the left sidebar, select **Repository > Commits**. 1. Select the title of the commit to display full information about the commit. diff --git a/doc/user/project/merge_requests/reviews/data_usage.md b/doc/user/project/merge_requests/reviews/data_usage.md index dd07f0b4a6e..f0eb3c015b6 100644 --- a/doc/user/project/merge_requests/reviews/data_usage.md +++ b/doc/user/project/merge_requests/reviews/data_usage.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, reference --- -# Suggested Reviewers Data Usage +# Suggested Reviewers Data Usage **(ULTIMATE SAAS)** ## How it works diff --git a/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg Binary files differdeleted file mode 100644 index e8aa4b7c730..00000000000 --- a/doc/user/project/merge_requests/reviews/img/add_another_suggestion_to_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg Binary files differdeleted file mode 100644 index d15f5d53e55..00000000000 --- a/doc/user/project/merge_requests/reviews/img/add_first_suggestion_to_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg Binary files differdeleted file mode 100644 index 3e1e9c20af9..00000000000 --- a/doc/user/project/merge_requests/reviews/img/apply_batch_of_suggestions_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/apply_suggestion_v13_9.png b/doc/user/project/merge_requests/reviews/img/apply_suggestion_v13_9.png Binary files differdeleted file mode 100644 index e27fa629672..00000000000 --- a/doc/user/project/merge_requests/reviews/img/apply_suggestion_v13_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png b/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png Binary files differdeleted file mode 100644 index 170c04542dd..00000000000 --- a/doc/user/project/merge_requests/reviews/img/custom_commit_v13_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/make_suggestion_v13_9.png b/doc/user/project/merge_requests/reviews/img/make_suggestion_v13_9.png Binary files differdeleted file mode 100644 index 92d5ba5ddda..00000000000 --- a/doc/user/project/merge_requests/reviews/img/make_suggestion_v13_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png b/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png Binary files differdeleted file mode 100644 index 80424d1f056..00000000000 --- a/doc/user/project/merge_requests/reviews/img/multi-line-suggestion-syntax.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg b/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg Binary files differdeleted file mode 100644 index fa8331effdb..00000000000 --- a/doc/user/project/merge_requests/reviews/img/remove_suggestion_from_batch_v13_1.jpg +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/suggestion_button_v13_9.png b/doc/user/project/merge_requests/reviews/img/suggestion_button_v13_9.png Binary files differdeleted file mode 100644 index 58e0508d8cf..00000000000 --- a/doc/user/project/merge_requests/reviews/img/suggestion_button_v13_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/img/suggestions_custom_commit_messages_v14_7.png b/doc/user/project/merge_requests/reviews/img/suggestions_custom_commit_messages_v14_7.png Binary files differdeleted file mode 100644 index 2805ef19f2d..00000000000 --- a/doc/user/project/merge_requests/reviews/img/suggestions_custom_commit_messages_v14_7.png +++ /dev/null diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 5291a845437..0de38874304 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, reference --- -# Review a merge request **(FREE)** +# Merge request reviews **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216054) in GitLab 13.5. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/245190) in GitLab 13.9. @@ -21,16 +21,19 @@ You can review merge requests from the GitLab interface. If you install the [GitLab Workflow VS Code extension](../../repository/vscode.md), you can also review merge requests in Visual Studio Code. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Merge request review](https://www.youtube.com/watch?v=2MayfXKpU08&list=PLFGfElNsQthYDx0A_FaNNfUm9NHsK6zED&index=183). + ## Suggested reviewers **(ULTIMATE SAAS)** -> [Introduced](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/review-recommender/-/epics/3) in GitLab 15.4. +> - [Introduced](https://gitlab.com/groups/gitlab-org/modelops/applied-ml/review-recommender/-/epics/3) in GitLab 15.4 as a [Beta](../../../../policy/alpha-beta-support.md#beta) feature [with a flag](../../../../administration/feature_flags.md) named `suggested_reviewers_control`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/368356) in GitLab 15.6. +> - Beta designation [removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113058) in GitLab 15.10. GitLab can suggest reviewers. Using the changes in a merge request and a project's contribution graph, machine learning suggestions appear in the reviewer section of the right sidebar. ![Suggested Reviewers](img/suggested_reviewers_v15_9.png) -This feature is currently in [Open Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#open-beta) behind a [feature flag](https://gitlab.com/gitlab-org/gitlab/-/issues/368356). - For more information, see [Data usage in Suggested Reviewers](data_usage.md). ### Enable suggested reviewers @@ -84,7 +87,7 @@ To download the changes included in a merge request as a diff: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**. 1. Select your merge request. -1. In the upper right, select **Code > Plain diff**. +1. In the upper-right corner, select **Code > Plain diff**. If you know the URL of the merge request, you can also download the diff from the command line by appending `.diff` to the URL. This example downloads the diff @@ -107,7 +110,7 @@ To download the changes included in a merge request as a patch file: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Merge requests**. 1. Select your merge request. -1. In the upper right, select **Code > Email patches**. +1. In the upper-right corner, select **Code > Patches**. If you know the URL of the merge request, you can also download the patch from the command line by appending `.patch` to the URL. This example downloads the patch @@ -172,7 +175,7 @@ If you have a review in progress, you can also add a comment from the **Overview When editing the **Reviewers** field in a new or existing merge request, GitLab displays the name of the matching [approval rule](../approvals/rules.md) -below the name of each suggested reviewer. [Code Owners](../../code_owners.md) are displayed as `Codeowner` without group detail. +below the name of each suggested reviewer. [Code Owners](../../codeowners/index.md) are displayed as `Codeowner` without group detail. This example shows reviewers and approval rules when creating a new merge request: @@ -234,7 +237,7 @@ When bulk-editing merge requests in a project, you can edit the following attrib To update multiple project merge requests at the same time: 1. In a project, go to **Merge requests**. -1. Select **Edit merge requests**. A sidebar on the right-hand side of your screen appears with +1. Select **Bulk edit**. A sidebar on the right-hand side of your screen appears with editable fields. 1. Select the checkboxes next to each merge request you want to edit. 1. Select the appropriate fields and their values from the sidebar. @@ -254,7 +257,7 @@ When bulk editing merge requests in a group, you can edit the following attribut To update multiple group merge requests at the same time: 1. In a group, go to **Merge requests**. -1. Select **Edit merge requests**. A sidebar on the right-hand side of your screen appears with +1. Select **Bulk edit**. A sidebar on the right-hand side of your screen appears with editable fields. 1. Select the checkboxes next to each merge request you want to edit. 1. Select the appropriate fields and their values from the sidebar. @@ -424,3 +427,4 @@ than 1000. The cached value is rounded to thousands (or millions) and updated ev ## Related topics - [Merge methods](../methods/index.md) +- [Draft Notes API](../../../../api/draft_notes.md) diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md index 668dece9fda..9187c5fad44 100644 --- a/doc/user/project/merge_requests/reviews/suggestions.md +++ b/doc/user/project/merge_requests/reviews/suggestions.md @@ -8,156 +8,176 @@ type: index, reference # Suggest changes **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25381) custom commit messages for suggestions in GitLab 13.9 [with a flag](../../../../administration/feature_flags.md) named `suggestions_custom_commit`. Disabled by default. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/297404) in GitLab 13.10. - -As a reviewer, you're able to suggest code changes with a Markdown syntax in merge request -diff threads. Then, the merge request author (or other users with appropriate -[permission](../../../permissions.md)) can apply these suggestions. -This action generates a commit in the merge request, authored by the user that suggested the changes. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/297404) in GitLab 13.10. Feature flag `suggestions_custom_commit` removed. + +Reviewers can suggest code changes with a Markdown syntax in merge request diff threads. +The merge request author (or other users with the appropriate role) can apply any or +all of the suggestions from the GitLab UI. Applying suggestions adds a commit to the +merge request, authored by the user who suggested the changes. + +## Create suggestions + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. On the secondary menu, select **Changes**. +1. Find the lines of code you want to change. + - To select a single line: + - Hover over the line number, and + select **Add a comment to this line** (**{comment}**). + - To select multiple lines: + 1. Hover over the line number, and select **Add a comment to this line** (**{comment}**). + 1. Select and drag your selection until all desired lines are included. To + learn more, see [Multi-line suggestions](#multi-line-suggestions). +1. In the comment toolbar, select **Insert suggestion** (**{doc-code}**). GitLab + inserts a pre-populated code block into your comment, like this: + + ````markdown + ```suggestion:-0+0 + The content of the line you selected is shown here. + ``` + ```` + +1. Edit the pre-populated code block to add your suggestion. +1. Select either **Start a review** or **Add to review** to add your comment to a + [review](index.md), or **Add comment now** to add the comment to the thread immediately. -1. Choose a line of code to be changed, add a new comment, then select - the **Insert suggestion** icon in the toolbar: +### Multi-line suggestions - ![Add a new comment](img/suggestion_button_v13_9.png) +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/232339) in GitLab 13.11: suggestions in multi-line comments also become multi-line. -1. In the comment, add your suggestion to the pre-populated code block: +When you review a merge request diff, you can propose changes to multiple lines (up to 200) +in a single suggestion, by either: - ![Add a suggestion into a code block tagged properly](img/make_suggestion_v13_9.png) +- Selecting and dragging, as described in [Create suggestions](#create-suggestions). + GitLab creates a suggestion block for you. +- Selecting a single line, then manually adjusting the range offsets. -1. Select either **Start a review** or **Add to review** to add your comment to a - [review](index.md), or **Add comment now** to add the comment to the thread immediately. +The range offsets in the first line of the suggestion describe line numbers relative +to the line you selected. The offsets specify the lines your suggestion intends to replace. +For example, this suggestion covers 3 lines above and 4 lines below the +commented line: - The suggestion in the comment can be applied by the merge request author - directly from the merge request: +````markdown +```suggestion:-3+4 + "--talk-name=ca.desrt.dconf", + "--socket=wayland", +``` +```` - ![Apply suggestions](img/apply_suggestion_v13_9.png) +When applied, the suggestion replaces from 3 lines above to 4 lines below the commented line: -1. Optionally specify a custom commit message for individual suggestions (GitLab 13.9 and later) to - describe your change. If you don't specify it, the default commit message is used. +![Multi-line suggestion preview](img/multi-line-suggestion-preview.png) - ![Custom commit](img/custom_commit_v13_9.png) +Suggestions for multiple lines are limited to 100 lines _above_ and 100 +lines _below_ the commented diff line. This allows for up to 200 changed lines per +suggestion. -After the author applies a suggestion: +## Apply suggestions -1. The suggestion is marked as **Applied**. -1. The thread is resolved. -1. GitLab creates a new commit with the changes. -1. If the user has the Developer role, GitLab pushes - the suggested change directly into the codebase in the merge request's branch. +Prerequisites: -## Multi-line suggestions +- You must be the author of the merge request, or have at least the Developer role in the project. -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/232339) in GitLab 13.11: suggestions in multi-line comments also become multi-line. +To apply suggested changes directly from the merge request: -Reviewers can also suggest changes to multiple lines with a single suggestion -within merge request diff threads by selecting and dragging selection to all -relevant line numbers or by adjusting the range offsets. The -offsets are relative to the position of the diff thread, and specify the -range to be replaced by the suggestion when it is applied. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. Find the comment containing the suggestion you want to apply. + - To apply suggestions individually, select **Apply suggestion**. + - To apply multiple suggestions in a single commit, select **Add suggestion to batch**. +1. Optional. Provide a custom commit message to describe your change. If you don't provide a custom message, the default commit message is used. +1. Select **Apply**. -![Multi-line suggestion syntax](img/multi-line-suggestion-syntax.png) +After a suggestion is applied: -In the previous example, the suggestion covers three lines above and four lines -below the commented line. When applied, it would replace from 3 lines _above_ -to 4 lines _below_ the commented line, with the suggested change. +- The suggestion is marked as **Applied**. +- The comment thread is resolved. +- GitLab creates a new commit with the changes. +- If the user has the Developer role, GitLab pushes + the suggested change directly into the codebase in the merge request's branch. -![Multi-line suggestion preview](img/multi-line-suggestion-preview.png) - -NOTE: -Suggestions for multiple lines are limited to 100 lines _above_ and 100 -lines _below_ the commented diff line. This allows for up to 200 changed lines per -suggestion. - -## Code block nested in suggestions +## Nest code blocks in suggestions To add a suggestion that includes a [fenced code block](../../../markdown.md#code-spans-and-blocks), wrap your suggestion in four backticks instead of three: -~~~markdown +`````markdown ````suggestion:-0+2 ```shell git config --global receive.advertisepushoptions true ``` ```` -~~~ +````` ![Output of a comment with a suggestion with a fenced code block](img/suggestion_code_block_output_v12_8.png) ## Configure the commit message for applied suggestions -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13086) in GitLab 12.7. +GitLab uses a default commit message when applying suggestions. This message +supports placeholders, and can be changed. For example, the default message +`Apply %{suggestions_count} suggestion(s) to %{files_count} file(s)` renders +like this if you apply three suggestions to two different files: -GitLab uses a default commit message -when applying suggestions: `Apply %{suggestions_count} suggestion(s) to %{files_count} file(s)` - -<!-- vale gitlab.BadPlurals = NO --> +```plaintext +Apply 3 suggestion(s) to 2 file(s) +``` -For example, consider that a user applied 3 suggestions to 2 different files, the -default commit message is: **Apply 3 suggestion(s) to 2 file(s)** +Merge requests created from forks use the template defined in the target project. -<!-- vale gitlab.BadPlurals = YES --> +To meet your project's needs, you can customize these messages and include other +placeholder variables: -These commit messages can be customized to follow any guidelines you might have. -To do so, expand the **Merge requests** tab within your project's **General** -settings and change the **Merge suggestions** text: +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. Scroll to **Merge suggestions**, and alter the text to meet your needs. + See [Supported variables](#supported-variables) for a list of placeholders + you can use in this message. -![Custom commit message for applied suggestions](img/suggestions_custom_commit_messages_v14_7.png) +### Supported variables -You can also use following variables besides static text: +The template for commit messages for applied suggestions supports these variables: | Variable | Description | Output example | |------------------------|-------------|----------------| | `%{branch_name}` | The name of the branch to which suggestions were applied. | `my-feature-branch` | -| `%{files_count}` | The number of files to which suggestions were applied.| **2** | +| `%{files_count}` | The number of files to which suggestions were applied.| `2` | | `%{file_paths}` | The paths of the file to which suggestions were applied. Paths are separated by commas.| `docs/index.md, docs/about.md` | | `%{project_path}` | The project path. | `my-group/my-project` | -| `%{project_name}` | The human-readable name of the project. | **My Project** | -| `%{suggestions_count}` | The number of suggestions applied.| **3** | +| `%{project_name}` | The human-readable name of the project. | `My Project` | +| `%{suggestions_count}` | The number of suggestions applied.| `3` | | `%{username}` | The username of the user applying suggestions. | `user_1` | -| `%{user_full_name}` | The full name of the user applying suggestions. | **User 1** | +| `%{user_full_name}` | The full name of the user applying suggestions. | `User 1` | For example, to customize the commit message to output -**Addresses user_1's review**, set the custom text to +`Addresses user_1's review`, set the custom text to `Addresses %{username}'s review`. -For merge requests created from forks, GitLab uses the template defined in target project. - -NOTE: -Custom commit messages for each applied suggestion is -introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/-/issues/25381). - ## Batch suggestions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [alpha feature](../../../../policy/alpha-beta-support.md#alpha-features) with a flag named `batch_suggestions`, disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [Experiment](../../../../policy/alpha-beta-support.md#experiment) with a flag named `batch_suggestions`, disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) in GitLab 13.2. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) in GitLab 13.11. [Feature flag `batch_suggestions`](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) removed. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326168) custom commit messages for batch suggestions in GitLab 14.4. -You can apply multiple suggestions at once to reduce the number of commits added -to your branch to address your reviewers' requests. - -1. To start a batch of suggestions to apply with a single commit, select **Add suggestion to batch**: - - ![A code change suggestion displayed, with the add-suggestion option highlighted.](img/add_first_suggestion_to_batch_v13_1.jpg "Add a suggestion to a batch") - -1. Add as many additional suggestions to the batch as you wish: - - ![A code change suggestion displayed, with the add-more suggestion option highlighted.](img/add_another_suggestion_to_batch_v13_1.jpg "Add another suggestion to a batch") - -1. To remove suggestions, select **Remove from batch**: - - ![A code change suggestion displayed, with the option to remove that suggestion from its batch highlighted.](img/remove_suggestion_from_batch_v13_1.jpg "Remove a suggestion from a batch") +To reduce the number of commits added to your branch, you can apply multiple +suggestions in a single commit. -1. Having added all the suggestions to your liking, when ready, select **Apply suggestions**. You - can optionally specify a custom commit message for [batch suggestions](#batch-suggestions) - (GitLab 14.4 and later) to describe your change. If you don't specify it, the default commit - message is used. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. For each suggestion you want to apply, and select **Add suggestion to batch**. +1. Optional. To remove a suggestion, select **Remove from batch**. +1. After you add your desired suggestions, select **Apply suggestions**. - ![A code change suggestion displayed, with the option to apply the batch of suggestions highlighted.](img/apply_batch_of_suggestions_v13_1.jpg "Apply a batch of suggestions") + WARNING: + If you apply a batch of suggestions containing changes from multiple authors, + you are credited as the resulting commit's author. If your project is configured + to [prevent approvals from users who add commits](../approvals/settings.md#prevent-approvals-by-users-who-add-commits), you are no longer an eligible + approver for this merge request. -WARNING: -Suggestions applied from multiple authors creates a commit authored by the user applying the suggestions. +1. Optional. Provide a custom commit message for [batch suggestions](#batch-suggestions) + (GitLab 14.4 and later) to describe your change. If you don't specify one, + the default commit message is used. ## Related topics diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 9f87f1e2e0d..075716e90c8 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -33,7 +33,14 @@ squash commits and merge commits. ## Set default squash options for a merge request Users with permission to create or edit a merge request can set the default squash options -for a merge request. To do this: +for a merge request. + +Prerequisites: + +- Your project must be [configured](#configure-squash-options-for-a-project) to allow or + encourage squashing. + +To do this: 1. Go to the merge request and select **Edit**. 1. Select or clear the **Squash commits when merge request is accepted** checkbox. @@ -58,6 +65,10 @@ squash the commits as part of the merge process: > - [Enabled on GitLab.com and self-managed by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39382) in GitLab 13.3. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/232536) in GitLab 13.8. Feature flag `squash_options` removed. +Prerequisites: + +- You must have at least the Maintainer role for this project. + To configure the default squashing behavior for all merge requests in your project: 1. On the top bar, select **Main menu > Projects** and find your project. diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index 62a2baa049b..0e339c65ed5 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -3,7 +3,6 @@ stage: Govern group: Compliance info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" type: reference, concepts -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/status_checks.html' --- # External status checks **(ULTIMATE)** @@ -12,6 +11,8 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/statu > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/320783) in GitLab 14.1. > - `failed` status [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329636) in GitLab 14.9. +Status checks are API calls to external systems that request the status of an external requirement. + You can create a status check that sends merge request data to third-party tools. When users create, change, or close merge requests, GitLab sends a notification. The users or automated workflows can then update the status of merge requests from outside of GitLab. @@ -70,7 +71,7 @@ using the API. You don't need to wait for a merge request webhook payload to be ## View the status checks on a project -Within each project's settings, you can see a list of status checks added to the project: +Within each project's settings, you can see a list of status check services added to the project: 1. In your project, go to **Settings > Merge requests** section. 1. Scroll down to **Status checks**. @@ -80,9 +81,9 @@ Within each project's settings, you can see a list of status checks added to the This list shows the service name, API URL, and targeted branch. It also provides actions to allow you to create, edit, or remove status checks. -## Add or update a status check +## Add or update a status check service -### Add a status check +### Add a status check service Within the **Status checks** sub-section, select the **Add status check** button. The **Add status check** form is then shown. @@ -91,7 +92,7 @@ The **Add status check** form is then shown. Filling in the form and selecting the **Add status check** button creates a new status check. -### Update a status check +### Update a status check service Within the **Status checks** sub-section, select the **Edit** button next to the status check you want to edit. @@ -134,7 +135,7 @@ for doesn't appear immediately. The search box requires If you want the status check to be applied to **all** merge requests, you can select the **All branches** option. -## Delete a status check +## Delete a status check service Within the **Status checks** sub-section, select the **Remove...** button next to the status check you want to delete. @@ -151,12 +152,15 @@ the status check and it **is not** recoverable. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327634) in GitLab 14.1. > - UI [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91504) in GitLab 15.2. > - Ability to retry failed external status checks [added](https://gitlab.com/gitlab-org/gitlab/-/issues/383200) in GitLab 15.8. +> - Widget [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111763) to poll for updates when there are pending status checks in GitLab 15.11. The status checks widget displays in merge requests and displays the following statuses: - **pending** (**{status-neutral}**), while GitLab waits for a response from an external status check. - **success** (**{status-success}**) or **failed** (**{status-failed}**), when GitLab receives a response from an external status check. +When there are pending status checks, the widget polls for updates every few seconds until it receives a **success** or **failed** response. + To retry a failed status check: 1. Expand the merge request widget to show the list of external status checks. diff --git a/doc/user/project/milestones/burndown_and_burnup_charts.md b/doc/user/project/milestones/burndown_and_burnup_charts.md index 81b334c0a02..c7ed6069cb6 100644 --- a/doc/user/project/milestones/burndown_and_burnup_charts.md +++ b/doc/user/project/milestones/burndown_and_burnup_charts.md @@ -137,14 +137,16 @@ To switch between the two settings, select either **Issues** or **Issue weight** When sorting by weight, make sure all your issues have weight assigned, because issues with no weight don't show on the chart. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Burndown and burnup charts do not show the correct issue status -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +A limitation of these charts is that [the days are in the UTC time zone](https://gitlab.com/gitlab-org/gitlab/-/issues/267967). + +This can cause the graphs to be inaccurate in other timezones. For example: + +- All the issues in a milestone are recorded as being closed on or before the last day. +- One issue was closed on the last day at 6 PM PST (Pacific time), which is UTC-7. +- The issue activity log displays the closure time at 6 PM on the last day of the milestone. +- The charts plot the time in UTC, so for this issue, the close time is 1 AM the following day. +- The charts show the milestone as incomplete and missing one closed issue. diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 5f9a2961df5..4641af262ca 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -65,7 +65,10 @@ Improving this experience is tracked in issue [339009](https://gitlab.com/gitlab You can view all the milestones you have access to in the entire GitLab namespace. You might not see some milestones because they're in projects or groups you're not a member of. -To do so, on the top bar select **Main menu > Milestones**. +To do so: + +1. On the top bar select **Main menu > Your work**. +1. On the left sidebar, select **Milestones**. ### View milestone details @@ -139,7 +142,7 @@ To edit a milestone: 1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. -1. Select **Edit**. +1. In the top right corner, select **Milestone actions** (**{ellipsis_v}**) and then select **Edit**. 1. Edit the title, start date, due date, or description. 1. Select **Save changes**. @@ -155,7 +158,7 @@ To edit a milestone: 1. On the top bar, select **Main menu > Projects** and find your project or **Main menu > Groups** and find your group. 1. Select a milestone's title. -1. Select **Delete**. +1. In the top right corner, select **Milestone actions** (**{ellipsis_v}**) and then select **Delete**. 1. Select **Delete milestone**. ## Promote a project milestone to a group milestone @@ -182,7 +185,7 @@ To promote a project milestone: 1. On the left sidebar, select **Issues > Milestones**. 1. Either: - Select **Promote to Group Milestone** (**{level-up}**) next to the milestone you want to promote. - - Select the milestone title, and then select **Promote**. + - Select the milestone title, and then select **Milestone actions** (**{ellipsis_v}**) > **Promote**. 1. Select **Promote Milestone**. ## Assign a milestone to an issue or merge request diff --git a/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png b/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png Binary files differnew file mode 100644 index 00000000000..50bcd1e8479 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/candidate_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/img/candidate_v15_7.png b/doc/user/project/ml/experiment_tracking/img/candidate_v15_7.png Binary files differdeleted file mode 100644 index fb2e2e706d6..00000000000 --- a/doc/user/project/ml/experiment_tracking/img/candidate_v15_7.png +++ /dev/null diff --git a/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png b/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png Binary files differnew file mode 100644 index 00000000000..3a2944c92ae --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/candidates_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/img/candidates_v15_7.png b/doc/user/project/ml/experiment_tracking/img/candidates_v15_7.png Binary files differdeleted file mode 100644 index 58dfe94a108..00000000000 --- a/doc/user/project/ml/experiment_tracking/img/candidates_v15_7.png +++ /dev/null diff --git a/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png b/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png Binary files differnew file mode 100644 index 00000000000..f7e25c1a0f1 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/experiments_v15_11.png diff --git a/doc/user/project/ml/experiment_tracking/img/experiments_v15_7.png b/doc/user/project/ml/experiment_tracking/img/experiments_v15_7.png Binary files differdeleted file mode 100644 index a7d4a3e559f..00000000000 --- a/doc/user/project/ml/experiment_tracking/img/experiments_v15_7.png +++ /dev/null diff --git a/doc/user/project/ml/experiment_tracking/index.md b/doc/user/project/ml/experiment_tracking/index.md index a7096d633a0..a2c2ab0cc40 100644 --- a/doc/user/project/ml/experiment_tracking/index.md +++ b/doc/user/project/ml/experiment_tracking/index.md @@ -4,78 +4,85 @@ group: Incubation info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. --- -# Machine Learning Experiment Tracking **(FREE)** +# Machine learning model experiments **(FREE)** -DISCLAIMER: -Machine Learning Experiment Tracking is an experimental feature being developed by the Incubation Engineering Department, -and will receive significant changes over time. This feature is being release with the aim of getting user feedback, but -is not stable and can lead to performance degradation. See below on how to disable this feature. +FLAG: +On self-managed GitLab, model experiment tracking is disabled by default. +To enable the feature, ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. +On GitLab.com, this feature is in private testing only. -When creating machine learning models, data scientists often experiment with different parameters, configurations, feature -engineering, and so on, to improve the performance of the model. Keeping track of all this metadata and the associated +NOTE: +Model experiment tracking is an [experimental feature](../../../../policy/alpha-beta-support.md). Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. + +When creating machine learning models, data scientists often experiment with different parameters, configurations, and feature +engineering to improve the performance of the model. Keeping track of all this metadata and the associated artifacts so that the data scientist can later replicate the experiment is not trivial. Machine learning experiment tracking enables them to log parameters, metrics, and artifacts directly into GitLab, giving easy access later on. -![List of Experiments](img/experiments_v15_7.png) +These features have been proposed: -![Experiment Candidates](img/candidates_v15_7.png) +- Searching experiments. +- Visual comparison of candidates. +- Creating, deleting, and updating candidates through the GitLab UI. -![Candidate Detail](img/candidate_v15_7.png) +For feature requests, see [epic 9341](https://gitlab.com/groups/gitlab-org/-/epics/9341). ## What is an experiment? -An experiment is a collection of comparable model candidates. Experiments can be long lived (for example, when they represent -a use case), or short lived (results from hyperparameter tuning triggered by a merge request), but usually hold model candidates -that have a similar set of parameters and metrics. +In a project, an experiment is a collection of comparable model candidates. +Experiments can be long-lived (for example, when they represent a use case), or +short-lived (results from hyperparameter tuning triggered by a merge request), +but usually hold model candidates that have a similar set of parameters measured +by the same metrics. + +![List of Experiments](img/experiments_v15_11.png) ## Model candidate A model candidate is a variation of the training of a machine learning model, that can be eventually promoted to a version -of the model. The goal of a data scientist is to find the model candidate whose parameter values lead to the best model +of the model. + +![Experiment Candidates](img/candidates_v15_11.png) + +The goal of a data scientist is to find the model candidate whose parameter values lead to the best model performance, as indicated by the given metrics. -Example parameters: +![Candidate Detail](img/candidate_v15_11.png) + +Some example parameters: -- Algorithm (linear regression, decision tree, and so on). +- Algorithm (such as linear regression or decision tree). - Hyperparameters for the algorithm (learning rate, tree depth, number of epochs). - Features included. -## Usage +## Track new experiments and candidates -### User access management +Experiment and trials can only be tracked through the +[MLflow](https://www.mlflow.org/docs/latest/tracking.html) client integration. +See [MLflow client integration](../../integrations/mlflow_client.md) for more information +on how to use GitLab as a backend for the MLflow Client. -An experiment is always associated to a project. Only users with access to the project an experiment is associated with -can view that experiment data. +## Explore model candidates -### Tracking new experiments and trials +Prerequisites: -Experiment and trials can only be tracked through the [MLFlow](https://www.mlflow.org/docs/latest/tracking.html) client -integration. More information on how to use GitLab as a backend for MLFlow Client can be found [at the documentation page](../../integrations/mlflow_client.md). +- You must have at least the Developer role to view experiment data. -### Exploring model candidates +To list the current active experiments, either go to `https/-/ml/experiments` or: -To list the current active experiments, navigate to `https/-/ml/experiments`. To display all trials -that have been logged, along with their metrics and parameters, select an experiment. To display details for a candidate, -select **Details**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Packages & registries > Model experiments**. +1. To display all candidates that have been logged, along with their metrics, parameters, and metadata, select an experiment. +1. To display details for a candidate, select **Details**. -### Logging artifacts +## View log artifacts Trial artifacts are saved as [generic packages](../../../packages/generic_packages/index.md), and follow all their -conventions. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the -package registry. The package name for a candidate is `ml_candidate_<candidate_id>`, with version `-`. The link to the -artifacts can also be accessed from the **Experiment Candidates** list or **Candidate detail**. - -### Limitations and future - -- Searching experiments, searching trials, visual comparison of trials, and creating, deleting and updating experiments and trials through GitLab UI is under development. - -## Disabling or enabling the Feature - -On self-managed GitLab, ML Experiment Tracking is disabled by default. To enable the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. -On GitLab.com, this feature is currently on private testing. - -## Feedback, roadmap and reports +limitations. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the +package registry. The package name for a candidate is `ml_experiment_<experiment_id>`, where the version is the candidate +IID. The link to the artifacts can also be accessed from the **Experiment Candidates** list or **Candidate detail**. -For updates on the development, refer to the [development epic](https://gitlab.com/groups/gitlab-org/-/epics/8560). +## Related topics -For feedback, bug reports and feature requests, refer to the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/381660). +- Development details in [epic 8560](https://gitlab.com/groups/gitlab-org/-/epics/8560). +- Add feedback in [issue 381660](https://gitlab.com/gitlab-org/gitlab/-/issues/381660). diff --git a/doc/user/project/organize_work_with_projects.md b/doc/user/project/organize_work_with_projects.md index 2b4ce6d2fd0..85a1dfda679 100644 --- a/doc/user/project/organize_work_with_projects.md +++ b/doc/user/project/organize_work_with_projects.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md index e55cf337d16..5cdf493fe6f 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md @@ -1,14 +1,11 @@ --- type: concepts -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# DNS records overview **(FREE)** - -_Read this document for a brief overview of DNS records in the scope -of GitLab Pages, for beginners in web development._ +# GitLab Pages DNS records **(FREE)** A Domain Name System (DNS) web service routes visitors to websites by translating domain names (such as `www.example.com`) into the @@ -74,7 +71,7 @@ Example: This way, visitors visiting `www.example.com` are redirected to `example.com`. -## MX record +## `MX` record MX records are used to define the mail exchanges that are used for the domain. This helps email messages arrive at your mail server correctly. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 24e9e6e15a4..a97fc1171fc 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -1,11 +1,10 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pages/getting_started_part_three.html' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Custom domains and SSL/TLS certificates **(FREE)** +# GitLab Pages custom domains **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238461) in GitLab 15.4, you can use verified domains to [bypass user email confirmation for SAML- or SCIM-provisioned users](../../../group/saml_sso/index.md#bypass-user-email-confirmation-with-verified-domains). @@ -13,7 +12,7 @@ You can use custom domains: - With GitLab Pages. - To [bypass user email confirmation for SAML- or SCIM-provisioned users](../../../group/saml_sso/index.md#bypass-user-email-confirmation-with-verified-domains). - When using custom domains this way, you use the GitLab Pages feature but can skip the [requirements](#requirements). + When using custom domains this way, you use the GitLab Pages feature but can skip the [prerequisites](#prerequisites). To use one or more custom domain names: @@ -24,7 +23,7 @@ To use one or more custom domain names: To set up Pages with a custom domain name, read the requirements and steps below. -### Requirements +### Prerequisites - A GitLab Pages website up and running, served under the default Pages domain (`*.gitlab.io`, for GitLab.com). @@ -34,7 +33,7 @@ To set up Pages with a custom domain name, read the requirements and steps below there are multiple DNS records on that name, you must use an `ALIAS` record. - A DNS `TXT` record to verify your domain's ownership. - Set either `external_http` or `external_https` in `/etc/gitlab/gitlab.rb` to the IP and port of - your [Pages Daemon](../../../../administration/pages/index.md#overview). + your [Pages daemon](../../../../administration/pages/index.md#the-gitlab-pages-daemon). If you don't have IPv6, you can omit the IPv6 address. Example: @@ -197,38 +196,6 @@ from the GitLab project. in place. Your domain is periodically reverified, and may be disabled if the record is removed. -##### Troubleshoot domain verification - -To manually verify that you have properly configured the domain verification -`TXT` DNS entry, you can run the following command in your terminal: - -```shell -dig _gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN> TXT -``` - -Expect the output: - -```plaintext -;; ANSWER SECTION: -_gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN>. 300 IN TXT "gitlab-pages-verification-code=<YOUR-VERIFICATION-CODE>" -``` - -In some cases it can help to add the verification code with the same domain name as you are trying to register. - -For a root domain: - -| From | DNS Record | To | -| ------------------------------------------------- | ---------- | ---------------------- | -| `example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | -| `_gitlab-pages-verification-code.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | - -For a subdomain: - -| From | DNS Record | To | -| ------------------------------------------------- | ---------- | ---------------------- | -| `www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | -| `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | - ### Add more domain aliases You can add more than one alias (custom domains and subdomains) to the same project. @@ -352,14 +319,36 @@ To enable this setting: If you use Cloudflare CDN in front of GitLab Pages, make sure to set the SSL connection setting to `full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). -<!-- ## Troubleshooting +## Troubleshooting + +### Domain verification + +To manually verify that you have properly configured the domain verification +`TXT` DNS entry, you can run the following command in your terminal: + +```shell +dig _gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN> TXT +``` + +Expect the output: + +```plaintext +;; ANSWER SECTION: +_gitlab-pages-verification-code.<YOUR-PAGES-DOMAIN>. 300 IN TXT "gitlab-pages-verification-code=<YOUR-VERIFICATION-CODE>" +``` + +In some cases it can help to add the verification code with the same domain name as you are trying to register. -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +For a root domain: -Each scenario can be a third-level heading, for example, `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +| From | DNS Record | To | +| ------------------------------------------------- | ---------- | ---------------------- | +| `example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | +| `_gitlab-pages-verification-code.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | + +For a subdomain: + +| From | DNS Record | To | +| ------------------------------------------------- | ---------- | ---------------------- | +| `www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | +| `_gitlab-pages-verification-code.www.example.com` | `TXT` | `gitlab-pages-verification-code=00112233445566778899aabbccddeeff` | diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 95ac2e50f29..91633e01ad2 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -1,12 +1,12 @@ --- type: reference description: "Automatic Let's Encrypt SSL certificates for GitLab Pages." -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab Pages integration with Let's Encrypt **(FREE)** +# GitLab Pages Let's Encrypt certificates **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/28996) in GitLab 12.1. @@ -21,7 +21,7 @@ open source Certificate Authority. WARNING: This feature covers only certificates for **custom domains**, not the wildcard certificate required to run [Pages daemon](../../../../administration/pages/index.md) **(FREE SELF)**. Wildcard certificate generation is tracked in [this issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3342). -## Requirements +## Prerequisites Before you can enable automatic provisioning of an SSL certificate for your domain, make sure you have: diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md index 398d8dc6e1e..484dc784fdb 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md @@ -1,14 +1,11 @@ --- type: concepts -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SSL/TLS certificates **(FREE)** - -_Read this document for a brief overview of SSL/TLS certificates in -the scope of GitLab Pages, for beginners in web development._ +# GitLab Pages SSL/TLS certificates **(FREE)** Every GitLab Pages project on GitLab.com is available under HTTPS for the default Pages domain (`*.gitlab.io`). Once you set diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md index f596e418b02..17618146350 100644 --- a/doc/user/project/pages/getting_started/pages_ci_cd_template.md +++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md @@ -1,11 +1,11 @@ --- type: reference, howto -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Create a Pages website by using a CI/CD template **(FREE)** +# Create a GitLab Pages website from a CI/CD template **(FREE)** GitLab provides `.gitlab-ci.yml` templates for the most popular Static Site Generators (SSGs). You can create your own `.gitlab-ci.yml` file from one of these templates, and run @@ -37,4 +37,4 @@ For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md index 509609e9b89..e8de80ac137 100644 --- a/doc/user/project/pages/getting_started/pages_forked_sample_project.md +++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md @@ -1,11 +1,11 @@ --- type: reference, howto -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Create a Pages website from a forked sample **(FREE)** +# Create a GitLab Pages website from a forked sample project **(FREE)** GitLab provides [sample projects for the most popular Static Site Generators (SSG)](https://gitlab.com/pages). You can fork one of the sample projects and run the CI/CD pipeline to generate a Pages website. @@ -18,8 +18,8 @@ configured to generate a Pages site. To fork a sample project and create a Pages website: 1. View the sample projects by navigating to the [GitLab Pages examples](https://gitlab.com/pages) group. -1. Select the name of the project you want to [fork](../../repository/forking_workflow.md#creating-a-fork). -1. In the upper right, select **Fork** and then choose a namespace to fork to. +1. Select the name of the project you want to [fork](../../repository/forking_workflow.md#create-a-fork). +1. In the upper-right corner, select **Fork**, then choose a namespace to fork to. 1. For your project, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. GitLab CI/CD builds and deploys your site. @@ -65,4 +65,4 @@ you can rename it to `<namespace>.gitlab.io`, where `<namespace>` is your GitLab ## Related topics -- [Download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts) +- [Download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts) diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index a3d6c8f75f9..c62ead69216 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -423,16 +423,11 @@ Now GitLab CI/CD not only builds the website, but also: - **Continuously deploys** every push to the `main` branch. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). ## Related topics -For more information, see the following blog posts. - -- Use GitLab CI/CD `environments` to - [deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/). -- Learn how to run jobs - [sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/). -- Learn [how to pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) - to deploy this website, <https://docs.gitlab.com>. -- Learn [how to use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/). +- [Deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) +- [Run jobs sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/) +- [Pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) +- [Use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) diff --git a/doc/user/project/pages/getting_started/pages_new_project_template.md b/doc/user/project/pages/getting_started/pages_new_project_template.md index a301d2b64be..cb1da3fb21f 100644 --- a/doc/user/project/pages/getting_started/pages_new_project_template.md +++ b/doc/user/project/pages/getting_started/pages_new_project_template.md @@ -1,10 +1,10 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Create a Pages website from a template **(FREE)** +# Create a GitLab Pages website from a project template **(FREE)** GitLab provides templates for the most popular Static Site Generators (SSGs). You can create a new project from a template and run the CI/CD pipeline to generate a Pages website. @@ -32,4 +32,4 @@ For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. To view the HTML and other assets that were created for the site, -[download the job artifacts](../../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +[download the job artifacts](../../../../ci/jobs/job_artifacts.md#download-job-artifacts). diff --git a/doc/user/project/pages/getting_started/pages_ui.md b/doc/user/project/pages/getting_started/pages_ui.md index 91c2c532a9a..00635fe6db2 100644 --- a/doc/user/project/pages/getting_started/pages_ui.md +++ b/doc/user/project/pages/getting_started/pages_ui.md @@ -4,7 +4,7 @@ group: Incubation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Create a Pages deployment for your static site **(FREE)** +# Create a GitLab Pages deployment for a static site **(FREE)** If you already have a GitLab project that contains your static site or framework, you can generate a GitLab Pages website from it. diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index a0c8073d6eb..6eb996db210 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -1,10 +1,10 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab Pages domain names, URLs, and base URLs **(FREE)** +# GitLab Pages default domain names and URLs **(FREE)** On this document, learn how to name your project for GitLab Pages according to your intended website's URL. diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 691757e3eca..a68ad604989 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -1,7 +1,7 @@ --- description: 'Learn how to use GitLab Pages to deploy a static website at no additional cost.' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -126,6 +126,13 @@ If you are running a self-managed instance of GitLab, <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a [video tutorial](https://www.youtube.com/watch?v=dD8c7WNcc6s) about how to get started with GitLab Pages administration. +### Configure GitLab Pages in a Helm Chart (Kubernetes) instance + +To configure GitLab Pages on instances deployed via Helm chart (Kubernetes), use either: + +- [The `gitlab-pages` subchart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-pages/). +- [An external GitLab Pages instance](https://docs.gitlab.com/charts/advanced/external-gitlab-pages/). + ## Security for GitLab Pages If your username is `example`, your GitLab Pages website is located at `example.gitlab.io`. diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 51c42eeecb8..05d0b461fea 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,10 +1,10 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Exploring GitLab Pages **(FREE)** +# GitLab Pages settings **(FREE)** This document is a user guide to explore the options and settings GitLab Pages offers. @@ -271,7 +271,7 @@ This problem most likely results from a missing `index.html` file in the public a 404 is encountered, confirm that the public directory contains an `index.html` file. If the file contains a different name such as `test.html`, the Pages site can still be accessed, but the full path would be needed. For example: `https//group-name.pages.example.com/project-name/test.html`. -The contents of the public directory can be confirmed by [browsing the artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts) from the latest pipeline. +The contents of the public directory can be confirmed by [browsing the artifacts](../../../ci/jobs/job_artifacts.md#download-job-artifacts) from the latest pipeline. Files listed under the public directory can be accessed through the Pages URL for the project. diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index 248a74a8abc..1b046d03f59 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md index 751db136e34..6ee8ea17aee 100644 --- a/doc/user/project/pages/public_folder.md +++ b/doc/user/project/pages/public_folder.md @@ -1,12 +1,12 @@ --- description: 'Learn how to configure the build output folder for the most common static site generators' -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configure the public files folder **(FREE)** +# GitLab Pages public folder **(FREE)** All the files that should be accessible by the browser must be in a root-level folder called `public`. @@ -91,14 +91,37 @@ NOTE: GitLab Pages supports only static sites. For Next.js, you can use Next's [Static HTML export functionality](https://nextjs.org/docs/advanced-features/static-html-export). -Use the `-o public` flag after `next export` as the build command, for -example: +With the release of [Next.js 13](https://nextjs.org/blog/next-13) a lot has changed on how Next.js works. +It is recommended to use the following `next.config.js` so all static assets can be exported properly: -```shell -next export -o public +```javascript +/** @type {import('next').NextConfig} */ +const nextConfig = { + reactStrictMode: true, + images: { + unoptimized: true, + }, + assetPrefix: "https://example.gitlab.io/namespace-here/my-gitlab-project/" +} + +module.exports = nextConfig +``` + +An example `.gitlab-ci.yml` can be as minimal as: + +```yaml +pages: + before_script: + - npm install + script: + - npm run build + - mv out/* public + artifacts: + paths: + - public ``` -### Nuxt.js +## Nuxt.js NOTE: GitLab Pages supports only static sites. @@ -146,7 +169,7 @@ module.exports = { ## Should you commit the `public` folder? Not necessarily. However, when the GitLab Pages deploy pipeline runs, it looks -for an [artifact](../../../ci/pipelines/job_artifacts.md) of that name. +for an [artifact](../../../ci/jobs/job_artifacts.md) of that name. If you set up a job that creates the `public` folder before deploy, such as by running `npm run build`, committing the folder isn't required. diff --git a/doc/user/project/pages/redirects.md b/doc/user/project/pages/redirects.md index f5447fd67ca..bd8206b3bda 100644 --- a/doc/user/project/pages/redirects.md +++ b/doc/user/project/pages/redirects.md @@ -1,10 +1,10 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Create redirects for GitLab Pages **(FREE)** +# GitLab Pages redirects **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/24) in GitLab Pages 1.25.0 and GitLab 13.4 behind a feature flag, disabled by default. > - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/367) in GitLab 13.5. diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index da53fe2f5e9..1d279436d2c 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -22,12 +22,14 @@ The [default branch](repository/branches/default.md) for your repository is prot ## Who can modify a protected branch +> Branch push permission [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118532) to require GitLab administrators to also have the **allowed** permission in GitLab 16.0. + When a branch is protected, the default behavior enforces these restrictions on the branch. | Action | Who can do it | |:-------------------------|:------------------------------------------------------------------| | Protect a branch | At least the Maintainer role. | -| Push to the branch | GitLab administrators and anyone with **Allowed** permission. (1) | +| Push to the branch | Anyone with **Allowed** permission. (1) | | Force push to the branch | No one. | | Delete the branch | No one. (2) | @@ -36,6 +38,40 @@ When a branch is protected, the default behavior enforces these restrictions on 1. No one can delete a protected branch using Git commands, however, users with at least Maintainer role can [delete a protected branch from the UI or API](#delete-a-protected-branch). +### When a branch matches multiple rules + +When a branch matches multiple rules, the **most permissive rule** determines the +level of protection for the branch. For example, consider these rules, which include +[wildcards](#configure-multiple-protected-branches-by-using-a-wildcard): + +| Branch name pattern | Allowed to merge | Allowed to push and merge | +|---------------------|------------------------|-----------------| +| `v1.x` | Maintainer | Maintainer | +| `v1.*` | Maintainer + Developer | Maintainer | +| `v*` | No one | No one | + +A branch named `v1.x` matches all three branch name patterns: `v1.x`, `v1.*`, and `v*`. +As the most permissive option determines the behavior, the resulting permissions for branch `v1.x` are: + +- **Allowed to merge:** Of the three settings, `Maintainer + Developer` is most permissive, + and controls branch behavior as a result. Even though the branch also matched `v1.x` and `v*` + (which each have stricter permissions), users with the Developer role can merge into the branch. +- **Allowed to push and merge:** Of the three settings, `Maintainer` is the most permissive, and controls + branch behavior as a result. Even though branches matching `v*` are set to `No one`, branches + that _also_ match `v1.x` or `v1.*` receive the more permissive `Maintainer` permission. + +To be certain that a rule controls the behavior of a branch, +_all_ other patterns that match must apply less or equally permissive rules. + +If you want to ensure that `No one` is allowed to push to branch `v1.x`, every pattern +that matches `v1.x` must set `Allowed to push and merge` to `No one`, like this: + +| Branch name pattern | Allowed to merge | Allowed to push and merge | +|---------------------|------------------------|-----------------| +| `v1.x` | Maintainer | No one | +| `v1.*` | Maintainer + Developer | No one | +| `v*` | No one | No one | + ### Set the default branch protection level Administrators can set a default branch protection level in the @@ -43,10 +79,41 @@ Administrators can set a default branch protection level in the ## Configure a protected branch +Configure protected branches for all projects in a group, or just for a project. + +### For all projects in a group **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106532) in GitLab 15.9 behind a feature flag, disabled by default. + +Group owners can create protected branches for a group. These settings are inherited by all projects in the group and can't be overridden by project settings. + +FLAG: +On self-managed GitLab, by default this feature is not available. +To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) +named `group_protected_branches`. On GitLab.com, this feature is not available. + +Prerequisite: + +- You must have the Owner role in the group. + +To protect a branch for all the projects in a group: + +1. On the top bar, select **Main menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Protected branches**. +1. In the **Branch** text box, type the branch name or a wildcard. +1. From the **Allowed to merge** list, select a role that can merge into this branch. +1. From the **Allowed to push and merge** list, select a role that can push to this branch. +1. Select **Protect**. + +The protected branch is added to the list of protected branches. + +### For a project + Prerequisite: - You must have at least the Maintainer role. -- When granting a group **Allowed to merge** or **Allowed to push** permissions +- When granting a group **Allowed to merge** or **Allowed to push and merge** permissions on a protected branch, the group must be added to the project. To protect a branch: @@ -56,7 +123,7 @@ To protect a branch: 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. -1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. +1. From the **Allowed to push and merge** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. 1. Select **Protect**. The protected branch displays in the list of protected branches. @@ -65,7 +132,7 @@ The protected branch displays in the list of protected branches. If both a specific rule and a wildcard rule apply to the same branch, the most permissive rule controls how the branch behaves. For merge controls to work properly, -set **Allowed to push** to a broader set of users than **Allowed to merge**. +set **Allowed to push and merge** to a broader set of users than **Allowed to merge**. Prerequisite: @@ -86,7 +153,7 @@ To protect multiple branches at the same time: | `*gitlab*` | `gitlab`, `gitlab/staging`, `master/gitlab/production` | 1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. -1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. +1. From the **Allowed to push and merge** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. 1. Select **Protect**. The protected branch displays in the list of protected branches. @@ -97,7 +164,7 @@ Users with at least the Developer role can create a protected branch. Prerequisites: -- **Allowed to push** is set to **No one** +- **Allowed to push and merge** is set to **No one** - **Allowed to merge** is set to **Developers**. You can create a protected branch by using the UI or API only. @@ -124,11 +191,7 @@ like the [GitLab workflow](../../topics/gitlab_flow.md). 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select **Developers + Maintainers**. -1. From the **Allowed to push** list, select **No one**. - - NOTE: - Setting a role, group or user as **Allowed to push** also allows those users to merge. - +1. From the **Allowed to push and merge** list, select **No one**. 1. Select **Protect**. ## Allow everyone to push directly to a protected branch @@ -139,7 +202,7 @@ You can allow everyone with write access to push to the protected branch. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. -1. From the **Allowed to push** list, select **Developers + Maintainers**. +1. From the **Allowed to push and merge** list, select **Developers + Maintainers**. 1. Select **Protect**. ## Allow deploy keys to push to a protected branch @@ -167,7 +230,7 @@ To allow a deploy key to push to a protected branch: 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. -1. From the **Allowed to push** list, select the deploy key. +1. From the **Allowed to push and merge** list, select the deploy key. 1. Select **Protect**. Deploy keys are not available in the **Allowed to merge** dropdown list. @@ -186,7 +249,7 @@ To protect a new branch and enable force push: 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. -1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. +1. From the **Allowed to push and merge** and **Allowed to merge** lists, select the settings you want. 1. To allow all users with push access to force push, turn on the **Allowed to force push** toggle. 1. To reject code pushes that change files listed in the `CODEOWNERS` file, turn on the **Require approval from code owners** toggle. @@ -203,10 +266,9 @@ Members who can push to this branch can now also force push. ## Require Code Owner approval on a protected branch **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13251) in GitLab 12.4. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in GitLab 13.5, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in GitLab 13.5, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. -For a protected branch, you can require at least one approval by a [Code Owner](code_owners.md). +For a protected branch, you can require at least one approval by a [Code Owner](codeowners/index.md). To protect a new branch and enable Code Owner's approval: @@ -214,7 +276,7 @@ To protect a new branch and enable Code Owner's approval: 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. -1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. +1. From the **Allowed to push and merge** and **Allowed to merge** lists, select the settings you want. 1. Turn on the **Require approval from code owners** toggle. 1. Select **Protect**. @@ -262,6 +324,12 @@ Protected branches can only be deleted by using GitLab either from the UI or API This prevents accidentally deleting a branch through local Git commands or third-party Git clients. +## Related topics + +- [Protected branches API](../../api/protected_branches.md) +- [Branches](repository/branches/index.md) +- [Branches API](../../api/branches.md) + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 152a55d24b7..3af475afa4f 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Protected tags **(FREE)** -Protected tags: +Protected [tags](repository/tags/index.md): - Allow control over who has permission to create tags. - Prevent accidental update or deletion once created. @@ -16,7 +16,7 @@ Each rule allows you to match either: - An individual tag name. - Wildcards to control multiple tags at once. -This feature evolved out of [protected branches](protected_branches.md) +This feature evolved out of [protected branches](protected_branches.md). ## Who can modify a protected tag @@ -27,22 +27,22 @@ By default: ## Configuring protected tags -To protect a tag, you must have at least the Maintainer role. +Prerequisites: -1. Go to the project's **Settings > Repository**. +- You must have at least the Maintainer role for the project. -1. From the **Tag** dropdown list, select the tag you want to protect or type and select **Create wildcard**. In the screenshot below, we chose to protect all tags matching `v*`: - - ![Protected tags page](img/protected_tags_page_v12_3.png) - -1. From the **Allowed to create** dropdown list, select users with permission to create - matching tags, and select **Protect**: - - ![Allowed to create tags dropdown list](img/protected_tags_permissions_dropdown_v12_3.png) - -1. After done, the protected tag displays in the **Protected tags** list: - - ![Protected tags list](img/protected_tags_list_v12_3.png) +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Protected tags**. +1. To protect a single tag, select **Tag**, then choose your tag from the dropdown list. +1. To protect all tags with names matching a string: + 1. Select **Tag**. + 1. Enter the string to use for tag matching. Wildcards (`*`) are supported. + 1. Select **Create wildcard**. +1. In **Allowed to create** , select either the users or roles that may create protected tags. +1. Select **Protect**. + +The protected tag (or wildcard) displays in the **Protected tags** list. ## Wildcard protected tags @@ -86,6 +86,32 @@ To prevent this problem: Users can still create branches, but not tags, with the protected names. +## Allow deploy keys to create protected tags + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325415) in GitLab 15.11. + +You can permit the owner of a [deploy key](deploy_keys/index.md) to create protected tags. +The deploy key works, even if the user isn't a member of the related project. However, the owner of the deploy +key must have at least read access to the project. + +Prerequisites: + +- The deploy key must be enabled for your project. A project deploy key is enabled by default when + it is created. However, a public deploy key must be + [granted](deploy_keys/index.md#grant-project-access-to-a-public-deploy-key) access to the + project. +- The deploy key must have [write access](deploy_keys/index.md#permissions) to your project + repository. + +To allow a deploy key to create a protected tag: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Protected tags**. +1. From the **Tag** dropdown list, select the tag you want to protect. +1. From the **Allowed to create** list, select the deploy key. +1. Select **Protect**. + ## Delete a protected tag You can manually delete protected tags with the GitLab API, or the @@ -106,6 +132,11 @@ Protected tags can only be deleted by using GitLab either from the UI or API. These protections prevent you from accidentally deleting a tag through local Git commands or third-party Git clients. +## Related topics + +- [Protected Tags API](../../api/protected_tags.md) +- [Tags API](../../api/tags.md) + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 90da47f7995..2c52a5d743a 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -49,113 +49,120 @@ The following quick actions are applicable to descriptions, discussions, and threads. Some quick actions might not be available to all subscription tiers. <!-- Keep this table sorted alphabetically --> - -| Command | Issue | Merge request | Epic | Action | -|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more active [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | -| `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. | -| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. | -| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. | -| `/assign_reviewer @user1 @user2` or `/reviewer @user1 @user2` or `/request_review @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users as reviewers. | -| `/assign_reviewer me` or `/reviewer me` or `/request_review me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself as a reviewer. | -| `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle emoji award. | +<!-- Don't prettify this table; it will expand out the last field into illegibility --> + +| Command | Issue | Merge request | Epic | Action | +|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------| +| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more active [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). +| `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. +| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. +| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. +| `/assign_reviewer @user1 @user2` or `/reviewer @user1 @user2` or `/request_review @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users as reviewers. +| `/assign_reviewer me` or `/reviewer me` or `/request_review me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself as a reviewer. +| `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle an emoji reaction. | `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line created a specific type of to-do item notification. | -| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | -| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | -| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | -| `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. | -| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | +| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). +| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. +| `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. +| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | `/confidential` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Mark issue or epic as confidential. Support for epics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213741) in GitLab 15.6. | -| `/copy_metadata <!merge_request>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another merge request in the project. | -| `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. | -| `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. | -| `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to do as done. | -| `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [draft status](merge_requests/drafts.md). Use for toggling the draft status ([deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92654) in GitLab 15.4.) | -| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. See [Chronic](https://gitlab.com/gitlab-org/ruby/gems/gitlab-chronic#examples) for more examples. | -| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue. Mark as a duplicate of, and related to, issue `<#issue>`. | -| `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. | -| `/estimate <time>` or `/estimate_time <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. For more information, see [Time tracking](time_tracking.md). Alias `/estimate_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | -| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. | -| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | -| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | -| `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | +| `/copy_metadata <!merge_request>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another merge request in the project. +| `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. +| `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. +| `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to-do item as done. +| `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [draft status](merge_requests/drafts.md). | +| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. See [Chronic](https://gitlab.com/gitlab-org/ruby/gems/gitlab-chronic#examples) for more examples. +| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue. Mark as a duplicate of, and related to, issue `<#issue>`. +| `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/estimate <time>` or `/estimate_time <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. For more information, see [Time tracking](time_tracking.md). Alias `/estimate_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). +| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. +| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). +| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. +| `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | `/link` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a link and description to [linked resources](../../operations/incident_management/linked_resources.md) in an incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/374964) in GitLab 15.5). | -| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). | -| `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. | -| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. | -| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | -| `/promote` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to epic. | -| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296787) in GitLab 14.5). In [GitLab 15.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/376760), you can also use the quick action when creating a new issue. | -| `/page <policy name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Start escalations for the incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79977) in GitLab 14.9). | -| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906) in GitLab 13.0) | -| `/ready` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [ready status](merge_requests/drafts.md#mark-merge-requests-as-ready) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90361) in GitLab 15.1). | -| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Replace current assignees with those specified. | -| `/reassign_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Replace current reviewers with those specified. | +| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). +| `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. +| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. +| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/promote` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to epic. +| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296787) in GitLab 14.5). In [GitLab 15.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/376760), you can also use the quick action when creating a new issue. +| `/page <policy name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Start escalations for the incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79977) in GitLab 14.9). +| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906) in GitLab 13.0) +| `/ready` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set the [ready status](merge_requests/drafts.md#mark-merge-requests-as-ready) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90361) in GitLab 15.1). +| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Replace current assignees with those specified. +| `/reassign_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Replace current reviewers with those specified. | `/rebase` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Rebase source branch. This schedules a background task that attempts to rebase the changes in the source branch on the latest commit of the target branch. If `/rebase` is used, `/merge` is ignored to avoid a race condition where the source branch is merged or deleted before it is rebased. If there are merge conflicts, GitLab displays a message that a rebase cannot be scheduled. Rebase failures are displayed with the merge request status. | -| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | -| `/relate #issue1 #issue2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Mark issues as related. | -| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | -| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | -| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. | -| `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. | -| `/remove_estimate` or `/remove_time_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. Alias `/remove_time_estimate` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | -| `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. | -| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | -| `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. | -| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue. | -| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | -| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Issue type must be `Incident`. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. | -| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. | -| `/spend <time> [<date>]` or `/spend_time <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. For more information, see [Time tracking](time_tracking.md). Alias `/spend_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. | -| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8041) in GitLab 12.7). | -| `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. | -| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | -| `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. | -| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | +| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. +| `/relate #issue1 #issue2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Mark issues as related. +| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. +| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). +| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. +| `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. +| `/remove_estimate` or `/remove_time_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. Alias `/remove_time_estimate` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). +| `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. +| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic. +| `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. +| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue. +| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. +| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Issue type must be `Incident`. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. +| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. +| `/spend <time> [<date>]` or `/spend_time <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. For more information, see [Time tracking](time_tracking.md). Alias `/spend_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. +| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review. +| `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. +| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. +| `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. +| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | `/timeline <timeline comment> \| <date(YYYY-MM-DD)> <time(HH:MM)>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a timeline event to this incident. For example, `/timeline DB load spiked \| 2022-09-07 09:30`. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368721) in GitLab 15.4). | -| `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. | -| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3 | -| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. | -| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all assignees. | +| `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. +| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3. +| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. +| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all assignees. | `/unassign_reviewer @user1 @user2` or `/remove_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific reviewers. -| `/unassign_reviewer me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove yourself as a reviewer. | -| `/unassign_reviewer` or `/remove_reviewer` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all reviewers. | -| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. | -| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. | -| `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. | -| `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. | -| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. | -| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a Zoom meeting to this issue or incident. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) users on GitLab Premium can add a short description when [adding a Zoom link to an incident](../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident).| +| `/unassign_reviewer me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove yourself as a reviewer. +| `/unassign_reviewer` or `/remove_reviewer` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all reviewers. +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. +| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. +| `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. +| `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. +| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. +| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a Zoom meeting to this issue or incident. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/230853) users on GitLab Premium can add a short description when [adding a Zoom link to an incident](../../operations/incident_management/linked_resources.md#link-zoom-meetings-from-an-incident). +| `/blocks <issue1> <issue2>` | **{check-circle}** Yes | **{check-circle}** No | **{dotted-circle}** No | Mark the issue as blocking other issues. The `<issue>` value should be in the format of `#issue`, `group/project#issue`, or the full issue URL. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214232) in GitLab 16.0). +| `/blocked_by <issue1> <issue2>` | **{check-circle}** Yes | **{check-circle}** No | **{dotted-circle}** No | Mark the issue as blocked by other issues. The `<issue>` value should be in the format of `#issue`, `group/project#issue`, or the full issue URL. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214232) in GitLab 16.0). ## Work items -The following quick actions can be applied through the description field when editing work items. - -| Command | Task | Objective | Key Result | Action | -|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | -| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | -| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | -| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. | -| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | -| `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line creates a specific type of to-do item notification. | -| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign one or more users. | -| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign yourself. | -| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove specific assignees. | -| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove all assignees. | -| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Replace current assignees with those specified. | -| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | -| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | -| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. | -| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. | -| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | -| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Remove due date. | -| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk`. | -| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). | -| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, and `2`. | -| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | +> Executing quick actions from comments [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391282) in GitLab 15.10. + +Work items in GitLab include [tasks](../tasks.md) and [OKRs](../okrs.md). +The following quick actions can be applied through the description field when editing or commenting on work items. + +| Command | Task | Objective | Key Result | Action +|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-------| +| `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. +| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. +| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. +| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. +| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. +| `/cc @user` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mention a user. In GitLab 15.0 and later, this command performs no action. You can instead type `CC @user` or only `@user`. [In GitLab 14.9 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/31200), mentioning a user at the start of a line creates a specific type of to-do item notification. +| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign one or more users. +| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Assign yourself. +| `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove specific assignees. +| `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** Yes | Remove all assignees. +| `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** Yes | Replace current assignees with those specified. +| `/label ~label1 ~label2` or `/labels ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. +| `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove specified labels. +| `/unlabel` or `/remove_label` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Remove all labels. +| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. +| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** Yes | Remove due date. +| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk`. +| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). +| `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, and `2`. +| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. +| `/type` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Converts work item to specified type. Available options for `<type>` include `Issue`, `Task`, `Objective` and `Key Result`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385227) in GitLab 16.0 [with a flag](../../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. ## Commit messages diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index dca34af41b4..d0c44b7e837 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -25,7 +25,7 @@ When you [create a release](#create-a-release): - GitLab automatically archives source code and associates it with the release. - GitLab automatically creates a JSON file that lists everything in the release, - so you can compare and audit releases. This file is called [release evidence](#release-evidence). + so you can compare and audit releases. This file is called [release evidence](release_evidence.md). When you create a release, or after, you can: @@ -45,8 +45,8 @@ To view a list of releases: ![Number of Releases](img/releases_count_v13_2.png "Incremental counter of Releases") - On public projects, this number is visible to all users. - - On private projects, this number is visible to users with Reporter - [permissions](../../permissions.md#project-members-permissions) or higher. + - On private projects, this number is visible to users with at least the Reporter + [role](../../permissions.md#project-members-permissions). ### Sort releases @@ -63,7 +63,7 @@ You can create a release: - [In the Releases page](#create-a-release-in-the-releases-page). - Using the [Releases API](../../../api/releases/index.md#create-a-release). -We recommend creating a release as one of the last steps in your CI/CD pipeline. +You should create a release as one of the last steps in your CI/CD pipeline. ### Create a release in the Releases page @@ -80,15 +80,16 @@ To create a release in the Releases page: - Select an existing Git tag. Selecting an existing tag that is already associated with a release results in a validation error. - Enter a new Git tag name. - 1. From the **Create from** dropdown list, select a branch or commit SHA to use when + 1. From the **Create tag** popover, select a branch or commit SHA to use when creating the new tag. 1. Optional. In the **Set tag message** text box, enter a message to create an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging#_annotated_tags). + 1. Select **Save**. 1. Optional. Enter additional information about the release, including: - [Title](release_fields.md#title). - [Milestones](#associate-milestones-with-a-release). - [Release notes](release_fields.md#release-notes-description). - - Whether or not to include the [Tag message](../../../topics/git/tags.md). + - Whether or not to include the [Tag message](../repository/tags/index.md). - [Asset links](release_fields.md#links). 1. Select **Create release**. @@ -179,7 +180,7 @@ release tag. When the `released_at` date and time has passed, the badge is autom You can create a release in the past using either the [Releases API](../../../api/releases/index.md#historical-releases) or the UI. When you set a past `released_at` date, an **Historical release** badge is displayed next to -the release tag. Due to being released in the past, [release evidence](#release-evidence) +the release tag. Due to being released in the past, [release evidence](release_evidence.md) is not available. ## Edit a release @@ -248,12 +249,12 @@ section, along with statistics about the issues in the milestones. Releases are also visible on the **Issues > Milestones** page, and when you select a milestone on this page. -Here is an example of milestones with no releases, one release, and two releases, respectively. +Here is an example of milestones with no releases, one release, and two releases. ![Milestones with and without Release associations](img/milestone_list_with_releases_v12_5.png) NOTE: -A subgroup's project releases cannot be associated with a supergroup's milestone. To learn +A subgroup's project releases cannot be associated with a parent group's milestone. To learn more, read issue #328054, [Releases cannot be associated with a supergroup milestone](https://gitlab.com/gitlab-org/gitlab/-/issues/328054). @@ -283,7 +284,7 @@ Deploy freezes help reduce uncertainty and risk when automating deployments. A maintainer can set a deploy freeze window in the user interface or by using the [Freeze Periods API](../../../api/freeze_periods.md) to set a `freeze_start` and a `freeze_end`, which are defined as [crontab](https://crontab.guru/) entries. -If the job that's executing is within a freeze period, GitLab CI/CD creates an environment +If the job that's executing is in a freeze period, GitLab CI/CD creates an environment variable named `$CI_DEPLOY_FREEZE`. To prevent the deployment job from executing, create a `rules` entry in your @@ -316,141 +317,6 @@ complete overlapping period. For more information, see [Deployment safety](../../../ci/environments/deployment_safety.md). -## Release evidence - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26019) in GitLab 12.6. - -Each time a release is created, GitLab takes a snapshot of data that's related to it. -This data is saved in a JSON file and called *release evidence*. The feature -includes test artifacts and linked milestones to facilitate -internal processes, like external audits. - -To access the release evidence, on the Releases page, select the link to the JSON file that's listed -under the **Evidence collection** heading. - -You can also [use the API](../../../api/releases/index.md#collect-release-evidence) to -generate release evidence for an existing release. Because of this, each release -can have multiple release evidence snapshots. You can view the release evidence and -its details on the Releases page. - -When the issue tracker is disabled, release evidence [can't be downloaded](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). - -Here is an example of a release evidence object: - -```json -{ - "release": { - "id": 5, - "tag_name": "v4.0", - "name": "New release", - "project": { - "id": 20, - "name": "Project name", - "created_at": "2019-04-14T11:12:13.940Z", - "description": "Project description" - }, - "created_at": "2019-06-28 13:23:40 UTC", - "description": "Release description", - "milestones": [ - { - "id": 11, - "title": "v4.0-rc1", - "state": "closed", - "due_date": "2019-05-12 12:00:00 UTC", - "created_at": "2019-04-17 15:45:12 UTC", - "issues": [ - { - "id": 82, - "title": "The top-right popup is broken", - "author_name": "John Doe", - "author_email": "john@doe.com", - "state": "closed", - "due_date": "2019-05-10 12:00:00 UTC" - }, - { - "id": 89, - "title": "The title of this page is misleading", - "author_name": "Jane Smith", - "author_email": "jane@smith.com", - "state": "closed", - "due_date": "nil" - } - ] - }, - { - "id": 12, - "title": "v4.0-rc2", - "state": "closed", - "due_date": "2019-05-30 18:30:00 UTC", - "created_at": "2019-04-17 15:45:12 UTC", - "issues": [] - } - ], - "report_artifacts": [ - { - "url":"https://gitlab.example.com/root/project-name/-/jobs/111/artifacts/download" - } - ] - } -} -``` - -### Collect release evidence **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in GitLab 12.10. - -When a release is created, release evidence is automatically collected. To initiate evidence collection any other time, use an [API call](../../../api/releases/index.md#collect-release-evidence). You can collect release evidence multiple times for one release. - -Evidence collection snapshots are visible on the Releases page, along with the timestamp the evidence was collected. - -### Include report artifacts as release evidence **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32773) in GitLab 13.2. - -When you create a release, if [job artifacts](../../../ci/yaml/index.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. - -Although job artifacts normally expire, artifacts included in release evidence do not expire. - -To enable job artifact collection you must specify both: - -1. [`artifacts:paths`](../../../ci/yaml/index.md#artifactspaths) -1. [`artifacts:reports`](../../../ci/yaml/index.md#artifactsreports) - -```yaml -ruby: - script: - - gem install bundler - - bundle install - - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml - artifacts: - paths: - - rspec.xml - reports: - junit: rspec.xml -``` - -If the pipeline ran successfully, when you create your release, the `rspec.xml` file is saved as -release evidence. - -If you [schedule release evidence collection](#schedule-release-evidence-collection), -some artifacts may already be expired by the time of evidence collection. To avoid this you can use -the [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) -keyword. For more information, see [issue 222351](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). - -### Schedule release evidence collection - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23697) in GitLab 12.8. - -In the API: - -- If you specify a future `released_at` date, the release becomes an **Upcoming release** - and the evidence is collected on the date of the release. You cannot collect - release evidence before then. -- If you specify a past `released_at` date, the release becomes an **Historical - release** and no evidence is collected. -- If you do not specify a `released_at` date, release evidence is collected on the - date the release is created. - ## Release permissions > Fixes to the permission model for create, update and delete actions [were introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327505) in GitLab 14.1. @@ -464,14 +330,15 @@ In the API: - Users with the Guest role have read and download access to the project releases. This includes associated Git-tag-names, release description, author information of the releases. - However, other repository-related information, such as [source code](release_fields.md#source-code), [release evidence](#release-evidence) are redacted. + However, other repository-related information, such as [source code](release_fields.md#source-code) and + [release evidence](release_evidence.md) are redacted. ### Publish releases without giving access to source code > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216485) in GitLab 15.6. Releases can be made accessible to non-project members while keeping repository-related information such as -[source code](release_fields.md#source-code) and [release evidence](#release-evidence) private. This is useful for +[source code](release_fields.md#source-code) and [release evidence](release_evidence.md) private. Use this for projects that use releases as a way to give access to new versions of software but do not want the source code to be public. @@ -527,4 +394,4 @@ See [the release permissions](#release-permissions) for more information. ### Note about storage -Note that the feature is built on top of Git tags, so virtually no extra data is needed besides to create the release itself. Additional assets and the release evidence that is automatically generated consume storage. +This feature is built on top of Git tags, so virtually no extra data is needed besides to create the release itself. Additional assets and the release evidence that is automatically generated consume storage. diff --git a/doc/user/project/releases/release_cicd_examples.md b/doc/user/project/releases/release_cicd_examples.md index 1ec82d6702e..7b5e808641c 100644 --- a/doc/user/project/releases/release_cicd_examples.md +++ b/doc/user/project/releases/release_cicd_examples.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/releases/release_cli.md b/doc/user/project/releases/release_cli.md index 5af19c7cced..bd3cfd26f1b 100644 --- a/doc/user/project/releases/release_cli.md +++ b/doc/user/project/releases/release_cli.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/releases/release_evidence.md b/doc/user/project/releases/release_evidence.md new file mode 100644 index 00000000000..3269bf8f44b --- /dev/null +++ b/doc/user/project/releases/release_evidence.md @@ -0,0 +1,140 @@ +--- +stage: Govern +group: Compliance +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Release evidence **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26019) in GitLab 12.6. + +Each time a release is created, GitLab takes a snapshot of data that's related to it. +This data is saved in a JSON file and called *release evidence*. The feature +includes test artifacts and linked milestones to facilitate +internal processes, like external audits. + +To access the release evidence, on the Releases page, select the link to the JSON file that's listed +under the **Evidence collection** heading. + +You can also [use the API](../../../api/releases/index.md#collect-release-evidence) to +generate release evidence for an existing release. Because of this, each release +can have multiple release evidence snapshots. You can view the release evidence and +its details on the Releases page. + +When the issue tracker is disabled, release evidence [can't be downloaded](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). + +Here is an example of a release evidence object: + +```json +{ + "release": { + "id": 5, + "tag_name": "v4.0", + "name": "New release", + "project": { + "id": 20, + "name": "Project name", + "created_at": "2019-04-14T11:12:13.940Z", + "description": "Project description" + }, + "created_at": "2019-06-28 13:23:40 UTC", + "description": "Release description", + "milestones": [ + { + "id": 11, + "title": "v4.0-rc1", + "state": "closed", + "due_date": "2019-05-12 12:00:00 UTC", + "created_at": "2019-04-17 15:45:12 UTC", + "issues": [ + { + "id": 82, + "title": "The top-right popup is broken", + "author_name": "John Doe", + "author_email": "john@doe.com", + "state": "closed", + "due_date": "2019-05-10 12:00:00 UTC" + }, + { + "id": 89, + "title": "The title of this page is misleading", + "author_name": "Jane Smith", + "author_email": "jane@smith.com", + "state": "closed", + "due_date": "nil" + } + ] + }, + { + "id": 12, + "title": "v4.0-rc2", + "state": "closed", + "due_date": "2019-05-30 18:30:00 UTC", + "created_at": "2019-04-17 15:45:12 UTC", + "issues": [] + } + ], + "report_artifacts": [ + { + "url":"https://gitlab.example.com/root/project-name/-/jobs/111/artifacts/download" + } + ] + } +} +``` + +## Collect release evidence **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in GitLab 12.10. + +When a release is created, release evidence is automatically collected. To initiate evidence collection any other time, use an [API call](../../../api/releases/index.md#collect-release-evidence). You can collect release evidence multiple times for one release. + +Evidence collection snapshots are visible on the Releases page, along with the timestamp the evidence was collected. + +## Include report artifacts as release evidence **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32773) in GitLab 13.2. + +When you create a release, if [job artifacts](../../../ci/yaml/index.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. + +Although job artifacts normally expire, artifacts included in release evidence do not expire. + +To enable job artifact collection you must specify both: + +1. [`artifacts:paths`](../../../ci/yaml/index.md#artifactspaths) +1. [`artifacts:reports`](../../../ci/yaml/index.md#artifactsreports) + +```yaml +ruby: + script: + - gem install bundler + - bundle install + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml + artifacts: + paths: + - rspec.xml + reports: + junit: rspec.xml +``` + +If the pipeline ran successfully, when you create your release, the `rspec.xml` file is saved as +release evidence. + +If you [schedule release evidence collection](#schedule-release-evidence-collection), +some artifacts may already be expired by the time of evidence collection. To avoid this you can use +the [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) +keyword. For more information, see [issue 222351](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). + +## Schedule release evidence collection + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23697) in GitLab 12.8. + +In the API: + +- If you specify a future `released_at` date, the release becomes an **Upcoming release** + and the evidence is collected on the date of the release. You cannot collect + release evidence before then. +- If you specify a past `released_at` date, the release becomes an **Historical + release** and no evidence is collected. +- If you do not specify a `released_at` date, release evidence is collected on the + date the release is created. diff --git a/doc/user/project/releases/release_fields.md b/doc/user/project/releases/release_fields.md index 120bbe32a63..d26ca87e384 100644 --- a/doc/user/project/releases/release_fields.md +++ b/doc/user/project/releases/release_fields.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -90,6 +90,8 @@ By default, GitLab fetches the release using `released_at` time. The use of the #### Permanent links to release assets +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375489) in GitLab 15.9, links for private releases can be accessed using a Personal Access Token. + The assets associated with a release are accessible through a permanent URL. GitLab always redirects this URL to the actual asset location, so even if the assets move to a different location, you can continue @@ -121,6 +123,14 @@ https://gitlab.com/gitlab-org/gitlab-runner/-/releases/v11.9.0-rc2/downloads/bin The physical location of the asset can change at any time and the direct link remains unchanged. +If the release is private, you need to provide a Personal Access Token with either `api` or `read_api` scopes using +a `private_token` query parameter or a `HTTP_PRIVATE_TOKEN` header when making the request. For example: + +```shell +curl --location --output filename "https://gitlab.example.com/my-group/my-project/-/releases/:release/downloads/:filepath?private_token=<your_access_token>" +curl --location --output filename --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/my-group/my-project/-/releases/:release/downloads/:filepath" +``` + #### Permanent links to latest release assets > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16821) in GitLab 14.9. @@ -245,7 +255,7 @@ release: ``` NOTE: -Directly attaching [job artifacts](../../../ci/pipelines/job_artifacts.md) +Directly attaching [job artifacts](../../../ci/jobs/job_artifacts.md) links to a release is not recommended, because artifacts are ephemeral and are used to pass data in the same pipeline. This means there's a risk that they could either expire or someone might manually delete them. diff --git a/doc/user/project/remote_development/connect_machine.md b/doc/user/project/remote_development/connect_machine.md new file mode 100644 index 00000000000..f981918c0ea --- /dev/null +++ b/doc/user/project/remote_development/connect_machine.md @@ -0,0 +1,109 @@ +--- +stage: Create +group: IDE +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tutorial: Connect a remote machine to the Web IDE **(FREE)** + +This tutorial shows you how to: + +- Create a development environment outside of GitLab. +- Connect a remote machine to the [Web IDE](../web_ide/index.md). + +To connect a remote machine to the Web IDE, you must: + +1. [Generate Let's Encrypt certificates](#generate-lets-encrypt-certificates). +1. [Connect a development environment to the Web IDE](#connect-a-development-environment-to-the-web-ide). + +## Prerequisites + +- A remote virtual machine with root access +- A domain address resolving to that machine +- Docker installation + +## Generate Let's Encrypt certificates + +To generate Let's Encrypt certificates: + +1. Create an `A` record to point a domain to your remote machine (for example, from `example.remote.gitlab.dev` to `1.2.3.4`). +1. Install [Certbot](https://certbot.eff.org/) to enable HTTPS: + + ```shell + sudo apt-get update + sudo apt-get install certbot + ``` + +1. Generate the certificates: + + ```shell + export EMAIL="YOUR_EMAIL@example.com" + export DOMAIN="example.remote.gitlab.dev" + + certbot -d "${DOMAIN}" \ + -m "${EMAIL}" \ + --config-dir ~/.certbot/config \ + --logs-dir ~/.certbot/logs \ + --work-dir ~/.certbot/work \ + --manual \ + --preferred-challenges dns certonly + ``` + +Now that you've generated the certificates, it's time to create and connect a development environment. + +## Connect a development environment to the Web IDE + +To connect a development environment to the Web IDE: + +1. Create a development environment: + + ```shell + export CERTS_DIR="/home/ubuntu/.certbot/config/live/${DOMAIN}" + export PROJECTS_DIR="/home/ubuntu" + + docker run -d \ + --name my-environment \ + -p 3443:3443 \ + -v "${CERTS_DIR}/fullchain.pem:/gitlab-rd-web-ide/certs/fullchain.pem" \ + -v "${CERTS_DIR}/privkey.pem:/gitlab-rd-web-ide/certs/privkey.pem" \ + -v "${PROJECTS_DIR}:/projects" \ + registry.gitlab.com/gitlab-org/remote-development/gitlab-rd-web-ide-docker:0.2-alpha \ + --log-level warn --domain "${DOMAIN}" --ignore-version-mismatch + ``` + + The new development environment starts automatically. + +1. Fetch a token: + + ```shell + docker exec my-environment cat TOKEN + ``` + +1. [Configure a remote connection](#configure-a-remote-connection). + +### Configure a remote connection + +To configure a remote connection from the Web IDE: + +1. Open the Web IDE. +1. On the menu bar, select **View > Terminal** or press <kbd>Control</kbd>+<kbd>`</kbd>. +1. In the terminal panel, select **Configure a remote connection**. +1. Enter the URL for the remote host including the port (for example, `yourdomain.com:3443`). +1. Enter the project path. +1. Enter the token you've fetched. + +Alternatively, you can pass the parameters from a URL and connect directly to the Web IDE: + +1. Run this command: + + ```shell + echo "https://gitlab-org.gitlab.io/gitlab-web-ide?remoteHost=${DOMAIN}:3443&hostPath=/projects" + ``` + +1. Go to that URL and enter the token you've fetched. + +You've done it! Your development environment now runs as a remote host that's connected to the Web IDE. + +## Related topics + +- [Manage a development environment](index.md#manage-a-development-environment) diff --git a/doc/user/project/remote_development/index.md b/doc/user/project/remote_development/index.md index 1abcca547db..d4156de2ebe 100644 --- a/doc/user/project/remote_development/index.md +++ b/doc/user/project/remote_development/index.md @@ -1,91 +1,59 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Remote Development **(FREE)** +# Remote development (Beta) **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.4 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.4 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/371084) in GitLab 15.7. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. WARNING: -This feature is in [Alpha](../../../policy/alpha-beta-support.md#alpha-features) and subject to change without notice. +This feature is in [Beta](../../../policy/alpha-beta-support.md#beta) and subject to change without notice. -DISCLAIMER: -This page contains information related to upcoming products, features, and functionality. -It is important to note that the information presented is for informational purposes only. -Please do not rely on this information for purchasing or planning purposes. -As with all projects, the items mentioned on this page are subject to change or delay. -The development, release, and timing of any products, features, or functionality remain at the -sole discretion of GitLab Inc. +You can use remote development to write and compile code hosted on GitLab. With remote development, you can: -You can use the [Web IDE](../web_ide/index.md) to commit changes to a project directly from your web browser without installing any dependencies or cloning any repositories. The Web IDE, however, lacks a native runtime environment on which you would compile code, run tests, or generate real-time feedback in the IDE. For a more complete IDE experience, you can pair the [Web IDE Beta](../web_ide_beta/index.md) with a Remote Development environment that has been properly configured to run as a host. +- Create a secure development environment in the cloud. +- Connect to that environment from your local machine through a web browser or client-based solution. -## Connect a remote machine to the Web IDE +## Web IDE as a frontend -Prerequisites: +You can use the [Web IDE](../web_ide/index.md) to make, commit, and push changes to a project directly from your web browser. +This way, you can update any project without having to install any dependencies or clone any repositories locally. -- A remote virtual machine with root access -- A domain address resolving to that machine -- Docker installation +The Web IDE, however, lacks a native runtime environment where you could compile code, run tests, or generate real-time feedback. +With remote development, you can use: -To connect a remote machine to the Web IDE, you must: +- The Web IDE as a frontend +- A separate machine as a backend runtime environment -1. [Generate Let's Encrypt certificates](#generate-lets-encrypt-certificates). -1. [Connect a development environment to the Web IDE](#connect-a-development-environment-to-the-web-ide). +For a complete IDE experience, connect the Web IDE to a development environment configured to run as a remote host. You can create this environment [inside](../../workspace/index.md) or [outside](connect_machine.md) of GitLab. -### Generate Let's Encrypt certificates +## Workspace -To generate Let's Encrypt certificates: +A [workspace](../../workspace/index.md) is a virtual sandbox environment for your code in GitLab that includes: -1. [Point a domain to your remote machine](#point-a-domain-to-your-remote-machine). -1. [Install Certbot](#install-certbot). -1. [Generate the certificates](#generate-the-certificates). +- A runtime environment +- Dependencies +- Configuration files -#### Point a domain to your remote machine +You can create a workspace from scratch or from a template that you can also customize. -To point a domain to your remote machine, create an `A` record from `example.remote.gitlab.dev` to `1.2.3.4`. +When you configure and connect a workspace to the [Web IDE](../web_ide/index.md), you can: -#### Install Certbot +- Edit files directly from the Web IDE and commit and push changes to GitLab. +- Use the Web IDE to run tests, debug code, and view real-time feedback. -[Certbot](https://certbot.eff.org/) is a free and open-source software tool that automatically uses Let's Encrypt certificates on manually administered websites to enable HTTPS. +## Manage a development environment -To install Certbot, run the following command: +### Create a development environment -```shell -sudo apt-get update -sudo apt-get install certbot -``` - -#### Generate the certificates - -```shell -export EMAIL="YOUR_EMAIL@example.com" -export DOMAIN="example.remote.gitlab.dev" - -certbot -d "${DOMAIN}" \ - -m "${EMAIL}" \ - --config-dir ~/.certbot/config \ - --logs-dir ~/.certbot/logs \ - --work-dir ~/.certbot/work \ - --manual \ - --preferred-challenges dns certonly -``` - -### Connect a development environment to the Web IDE - -To connect a development environment to the Web IDE: - -1. [Create a development environment](#manage-a-development-environment). -1. [Fetch a token](#fetch-a-token). -1. [Configure a remote connection](#configure-a-remote-connection). - -#### Manage a development environment - -**Create a development environment** +To create a development environment, run this command: ```shell export CERTS_DIR="/home/ubuntu/.certbot/config/live/${DOMAIN}" @@ -97,64 +65,37 @@ docker run -d \ -v "${CERTS_DIR}/fullchain.pem:/gitlab-rd-web-ide/certs/fullchain.pem" \ -v "${CERTS_DIR}/privkey.pem:/gitlab-rd-web-ide/certs/privkey.pem" \ -v "${PROJECTS_DIR}:/projects" \ - registry.gitlab.com/gitlab-com/create-stage/editor-poc/remote-development/gitlab-rd-web-ide-docker:0.1-alpha \ + registry.gitlab.com/gitlab-org/remote-development/gitlab-rd-web-ide-docker:0.2-alpha \ --log-level warn --domain "${DOMAIN}" --ignore-version-mismatch ``` The new development environment starts automatically. -**Stop a development environment** +### Stop a development environment + +To stop a running development environment, run this command: ```shell docker container stop my-environment ``` -**Start a development environment** +### Start a development environment + +To start a stopped development environment, run this command: ```shell docker container start my-environment ``` -The token changes every time you restart the development environment. +The token changes every time you start the development environment. -**Remove a development environment** +### Remove a development environment To remove a development environment: -1. Stop the development environment. -1. Run the following command: +1. [Stop the development environment](#stop-a-development-environment). +1. Run this command: ```shell docker container rm my-environment ``` - -#### Fetch a token - -```shell -docker exec my-environment cat TOKEN -``` - -#### Configure a remote connection - -To configure a remote connection from the Web IDE: - -1. Open the Web IDE. -1. In the Menu Bar, select **View > Terminal** or press <kbd>Control</kbd>+<kbd>`</kbd>. -1. In the terminal panel, select **Configure a remote connection**. -1. Enter the URL for the remote host including the port (for example, `yourdomain.com:3443`). -1. Enter the project path. -1. Enter the [token you fetched](#fetch-a-token). - -Alternatively, you can pass the parameters from a URL and connect directly to the Web IDE: - -1. Run the following command: - - ```shell - echo "https://gitlab-org.gitlab.io/gitlab-web-ide?remoteHost=${DOMAIN}:3443&hostPath=/projects" - ``` - -1. Go to that URL and enter the [token you fetched](#fetch-a-token). - -## Related topics - -- [Web IDE Beta](../web_ide_beta/index.md) diff --git a/doc/user/project/repository/branches/default.md b/doc/user/project/repository/branches/default.md index b33dc4450a9..e58cc0bf6a4 100644 --- a/doc/user/project/repository/branches/default.md +++ b/doc/user/project/repository/branches/default.md @@ -134,7 +134,7 @@ groups and subgroups can override this instance-wide setting for their projects. Instance-level protections for default branches can be overridden on a per-group basis by the group's owner. In -[GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can +[GitLab Premium or Ultimate](https://about.gitlab.com/pricing/), GitLab administrators can disable this privilege for group owners, enforcing the instance-level protection rule: 1. On the top bar, select **Main menu > Admin**. @@ -153,7 +153,7 @@ GitLab administrators can still update the default branch protection of a group. Instance-level protections for [default branch](#default-branch) can be overridden on a per-group basis by the group's owner. In -[GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can +[GitLab Premium or Ultimate](https://about.gitlab.com/pricing/), GitLab administrators can [enforce protection of initial default branches](#prevent-overrides-of-default-branch-protection) which locks this setting for group owners. diff --git a/doc/user/project/repository/branches/img/branch_filter_search_box_v13_12.png b/doc/user/project/repository/branches/img/branch_filter_search_box_v13_12.png Binary files differdeleted file mode 100644 index a1cf9f10122..00000000000 --- a/doc/user/project/repository/branches/img/branch_filter_search_box_v13_12.png +++ /dev/null diff --git a/doc/user/project/repository/branches/img/compare_branches_v13_12.png b/doc/user/project/repository/branches/img/compare_branches_v13_12.png Binary files differdeleted file mode 100644 index 29627406729..00000000000 --- a/doc/user/project/repository/branches/img/compare_branches_v13_12.png +++ /dev/null diff --git a/doc/user/project/repository/branches/img/repository_filter_search_box_v13_12.png b/doc/user/project/repository/branches/img/repository_filter_search_box_v13_12.png Binary files differdeleted file mode 100644 index 230abf0d875..00000000000 --- a/doc/user/project/repository/branches/img/repository_filter_search_box_v13_12.png +++ /dev/null diff --git a/doc/user/project/repository/branches/img/swap_revisions_after_v13_12.png b/doc/user/project/repository/branches/img/swap_revisions_after_v13_12.png Binary files differdeleted file mode 100644 index 7eb10d10938..00000000000 --- a/doc/user/project/repository/branches/img/swap_revisions_after_v13_12.png +++ /dev/null diff --git a/doc/user/project/repository/branches/img/swap_revisions_before_v13_12.png b/doc/user/project/repository/branches/img/swap_revisions_before_v13_12.png Binary files differdeleted file mode 100644 index f92c4279871..00000000000 --- a/doc/user/project/repository/branches/img/swap_revisions_before_v13_12.png +++ /dev/null diff --git a/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png b/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png Binary files differnew file mode 100644 index 00000000000..8472015c21b --- /dev/null +++ b/doc/user/project/repository/branches/img/view_branch_protections_v15_10.png diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 60504b94502..7c09ce5c580 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -6,142 +6,260 @@ info: "To determine the technical writer assigned to the Stage/Group associated # Branches **(FREE)** -A branch is a version of a project's working tree. You create a branch for each -set of related changes you make. This keeps each set of changes separate from -each other, allowing changes to be made in parallel, without affecting each -other. +Branches are versions of a project's working tree. When you create a new +[project](../../index.md), GitLab creates a [default branch](default.md) (which +cannot be deleted) for your repository. Default branch settings can be configured +at the project, subgroup, group, or instance level. -After pushing your changes to a new branch, you can: +As your project grows, your team [creates](../web_editor.md#create-a-branch) more +branches, preferably by following [branch naming patterns](#prefix-branch-names-with-issue-numbers). +Each branch represents a set of changes, which allows development work to be done +in parallel. Development work in one branch does not affect another branch. -- Create a [merge request](../../merge_requests/index.md). You can streamline this process - by following [branch naming patterns](#naming). -- Perform inline code review. -- [Discuss](../../../discussions/index.md) your implementation with your team. -- Preview changes submitted to a new branch with [Review Apps](../../../../ci/review_apps/index.md). +Branches are the foundation of development in a project: -You can also request [approval](../../merge_requests/approvals/index.md) -from your managers. +1. To get started, create a branch and add commits to it. +1. When the work is ready for review, create a [merge request](../../merge_requests/index.md) to propose + merging the changes in your branch. To streamline this process, you should follow + [branch naming patterns](#prefix-branch-names-with-issue-numbers). +1. Preview changes in a branch with a [review app](../../../../ci/review_apps/index.md). +1. After the contents of your branch are merged, [delete the merged branch](#delete-merged-branches). -For more information on managing branches using the GitLab UI, see: +## Create branch -- [Default branches](default.md): When you create a new [project](../../index.md), GitLab creates a - default branch for the repository. You can change this setting at the project, - subgroup, group, or instance level. -- [Create a branch](../web_editor.md#create-a-branch) -- [Protected branches](../../protected_branches.md#protected-branches) -- [Delete merged branches](#delete-merged-branches) -- [Branch filter search box](#branch-filter-search-box) +To create a new branch from the GitLab UI: -You can also manage branches using the -[command line](../../../../gitlab-basics/start-using-git.md#create-a-branch). +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. +1. On the top right, select **New branch**. +1. Enter a **Branch name**. +1. In **Create from**, select the base of your branch: an existing branch, an existing + tag, or a commit SHA. +1. Select **Create branch**. -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>Watch the video [GitLab Flow](https://www.youtube.com/watch?v=InKNIvky2KE). +### In a blank project -See also: +A [blank project](../../index.md#create-a-blank-project) does not contain a branch, but +you can add one. -- [Branches API](../../../../api/branches.md), for information on operating on repository branches using the GitLab API. -- [GitLab Flow](../../../../topics/gitlab_flow.md) documentation. -- [Getting started with Git](../../../../topics/git/index.md) and GitLab. +Prerequisites: -## Naming +- You must have at least the Developer role in the project. +- Unless you have the Maintainer or Owner roles, the + [default branch protection](../../../group/manage.md#change-the-default-branch-protection-of-a-group) + must be set to `Partially protected` or `Not protected` for you to push a commit + to the default branch. -Prefix a branch name with an issue number to streamline merge request creation. -When you create a merge request for a branch with a name beginning with an issue -number, GitLab: +To add a [default branch](default.md) to an empty project: -- Marks the issue as related. If your project is configured with a - [default closing pattern](../../issues/managing_issues.md#default-closing-pattern), - merging this merge request [also closes](../../issues/managing_issues.md#closing-issues-automatically) - the related issue. -- Copies label and milestone metadata from the issue. +1. On the top bar, select **Main menu > Projects** and find your project. +1. Scroll to **The repository for this project is empty** and select the type of + file you want to add. +1. In the Web IDE, make any desired changes to this file, then select **Create commit**. +1. Enter a commit message, and select **Commit**. -## Compare +GitLab creates a default branch and adds your file to it. -To compare branches in a repository: +### From an issue -1. Navigate to your project's repository. -1. Select **Repository > Compare** in the sidebar. -1. Select the target repository to compare with the [repository filter search box](#repository-filter-search-box). -1. Select branches to compare using the [branch filter search box](#branch-filter-search-box). -1. Select **Compare** to view the changes inline: +When viewing an issue, you can create an associated branch directly from that page. - ![compare branches](img/compare_branches_v13_12.png) +Prerequisites: -## Delete merged branches +- You must have at least the Developer role in the project. -![Delete merged branches](img/delete_merged_branches.png) +To create a branch from an issue: -This feature allows merged branches to be deleted in bulk. Only branches that -have been merged into the project's default branch and -[are not protected](../../protected_branches.md) are deleted as part of -this operation. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues** (**{issues}**) and find your issue. +1. Below the issue description, find the **Create merge request** dropdown list, and select + **{chevron-down}** to display the dropdown list. +1. Select **Create branch**. A default **Branch name** is provided, based on the + [default pattern](#configure-default-pattern-for-branch-names-from-issues) for + this project. If desired, enter a different **Branch name**. +1. Select **Create branch** to create the branch based on your project's + [default branch](default.md). -It's particularly useful to clean up old branches that were not deleted -automatically when a merge request was merged. +## Manage and protect branches + +GitLab provides you multiple methods to protect individual branches. These methods +ensure your branches receive oversight and quality checks from their creation to their deletion: -## Repository filter search box +- The [default branch](default.md) in your project receives extra protection. +- Configure [protected branches](../../protected_branches.md#protected-branches) + to restrict who can commit to a branch, merge other branches into it, or merge + the branch itself into another branch. +- Configure [approval rules](../../merge_requests/approvals/rules.md) to set review + requirements, including [security-related approvals](../../merge_requests/approvals/rules.md#security-approvals), before a branch can merge. +- Integrate with third-party [status checks](../../merge_requests/status_checks.md) + to ensure your branch contents meet your standards of quality. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52967) in GitLab 13.10. +You can manage your branches: -This feature allows you to search and select a repository quickly when [comparing branches](#compare). +- With the GitLab user interface. +- With the [command line](../../../../gitlab-basics/start-using-git.md#create-a-branch). +- With the [Branches API](../../../../api/branches.md). -![Repository filter search box](img/repository_filter_search_box_v13_12.png) +### View all branches -Search results appear in the following order: +To view and manage your branches in the GitLab user interface: -- Repositories with names exactly matching the search terms. -- Other repositories with names that include search terms, sorted alphabetically. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. -## Branch filter search box +On this page, you can: -![Branch filter search box](img/branch_filter_search_box_v13_12.png) +- See all branches, active branches, or stale branches. +- Create new branches. +- [Compare branches](#compare-branches). +- Delete merged branches. -This feature allows you to search and select branches quickly. Search results appear in the following order: +### View branches with configured protections -- Branches with names that matched search terms exactly. -- Other branches with names that include search terms, sorted alphabetically. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88279) in GitLab 15.1 with a flag named `branch_rules`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/363170) in GitLab 15.10. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363170) in GitLab 15.11 -Sometimes when you have hundreds of branches you may want a more flexible matching pattern. In such cases you can use the following operators: +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../feature_flags.md) named `branch_rules`. +On GitLab.com, this feature is available. -- `^` matches beginning of branch name, for example `^feat` would match `feat/user-authentication` -- `$` matches end of branch name, for example `widget$` would match `feat/search-box-widget` -- `*` wildcard matcher, for example `branch*cache*` would match `fix/branch-search-cache-expiration` +Branches in your repository can be [protected](../../protected_branches.md) in multiple ways. You can: -These operators can be mixed, for example `^chore/*migration$` would match `chore/user-data-migration` +- Limit who can push to the branch. +- Limit who can merge the branch. +- Require approval of all changes. +- Require external tests to pass. -## Swap revisions +The **Branch rules overview** page shows all branches with any configured protections, +and their protection methods: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60491) in GitLab 13.12. +![Example of a branch with configured protections](img/view_branch_protections_v15_10.png) -![Before swap revisions](img/swap_revisions_before_v13_12.png) +Prerequisites: -The Swap revisions feature allows you to swap the Source and Target revisions. When the Swap revisions button is selected, the selected revisions for Source and Target is swapped. +- You must have at least the Maintainer role in the project. -![After swap revisions](img/swap_revisions_after_v13_12.png) +To view the **Branch rules overview** list: -## View branches with configured protections **(FREE SELF)** +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Branch Rules** to view all branches with protections. + - To add protections to a new branch: + 1. Select **Add branch rule**. + 1. Select **Create protected branch**. + - To view more information about protections on an existing branch: + 1. Identify the branch you want more information about. + 1. Select **Details** to see information about its: + - [Branch protections](../../protected_branches.md). + - [Approval rules](../../merge_requests/approvals/rules.md). + - [Status checks](../../merge_requests/status_checks.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88279) in GitLab 15.1 with a flag named `branch_rules`. Disabled by default. +## Name your branch -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../feature_flags.md) named `branch_rules`. -On GitLab.com, this feature is not available. -This feature is not ready for production use. +Git enforces [branch name rules](https://git-scm.com/docs/git-check-ref-format) +to help ensure branch names remain compatible with other tools. GitLab +adds extra requirements for branch names, and provides benefits for well-structured branch names. -Branches in your repository can be [protected](../../protected_branches.md) by limiting -who can push to a branch, require approval for those pushed changes, or merge those changes. -To help you track the protections for all branches, the **Branch rules overview** -page shows your branches with their configured rules. +GitLab enforces these additional rules on all branches: -To view the **Branch rules overview** list: +- No spaces are allowed in branch names. +- Branch names with 40 hexadecimal characters are prohibited, because they are similar to Git commit hashes. + +Common software packages, like Docker, may enforce +[additional branch naming restrictions](../../../../administration/packages/container_registry.md#docker-connection-error). + +For best compatibility with other software packages, use only numbers, hyphens (`-`), +underscores (`_`), and lower-case letters from the ASCII standard table. You +can use forward slashes (`/`) and emoji in branch names, but compatibility with other +software packages cannot be guaranteed. + +Branch names with specific formatting offer extra benefits: + +- Streamline your merge request workflow by + [prefixing branch names with issue numbers](#prefix-branch-names-with-issue-numbers). +- Automate [branch protections](../../protected_branches.md) based on branch name. +- Test branch names with [push rules](../push_rules.md) before branches are pushed up to GitLab. +- Define which [CI/CD jobs](../../../../ci/jobs/index.md) to run on merge requests. + +### Configure default pattern for branch names from issues + +By default, GitLab uses the pattern `%{id}-%{title}` when creating a branch from +an issue, but you can change this pattern. + +Prerequisites: + +- You must have at least the Maintainer role for the project. + +To change the default pattern for branches created from issues: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Repository**. -1. Expand **Branch Rules** to view all branches with protections. -1. Select **Details** next to your desired branch to show information about its: - - [Branch protections](../../protected_branches.md). - - [Approval rules](../../merge_requests/approvals/rules.md). - - [Status checks](../../merge_requests/status_checks.md). +1. On the left sidebar, select **Settings > Repository** and expand **Branch defaults**. +1. Scroll to **Branch name template** and enter a value. The field supports these variables: + - `%{id}`: The numeric ID of the issue. + - `%{title}`: The title of the issue, modified to use only characters acceptable in Git branch names. +1. Select **Save changes**. + +### Prefix branch names with issue numbers + +To streamline the creation of merge requests, start your branch name with an +issue number. GitLab uses the issue number to import data into the merge request: + +- The issue is marked as related to the merge request. The issue and merge request + display links to each other. +- The branch is connected to the issue. +- If your project is configured with a + [default closing pattern](../../issues/managing_issues.md#default-closing-pattern), + merging the merge request [also closes](../../issues/managing_issues.md#closing-issues-automatically) + the related issue. +- Issue milestone and labels are copied to the merge request. + +## Compare branches + +> - Repository filter search box [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52967) in GitLab 13.10. +> - Revision swapping [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60491) in GitLab 13.12. + +The default compare mode uses the `git diff from...to` method, instead of the +`git diff from to` method, to compare branches. The `git diff from...to` method +provides a more human-readable diff, because it does not include unrelated changes +made to the target branch after the source branch was created. + +NOTE: +The `git diff from...to` method shows all changes from a cherry-picked commit as +new changes. It uses the merge base, not the actual commit content, to compare branches. + +To compare branches in a repository: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Compare revisions**. +1. Select the **Source** branch to search for your desired branch. Exact matches are + shown first. You can refine your search with operators: + - `^` matches the beginning of the branch name: `^feat` matches `feat/user-authentication`. + - `$` matches the end of the branch name: `widget$` matches `feat/search-box-widget`. + - `*` matches using a wildcard: `branch*cache*` matches `fix/branch-search-cache-expiration`. + - You can combine operators: `^chore/*migration$` matches `chore/user-data-migration`. +1. Select the **Target** repository and branch. Exact matches are shown first. +1. Select **Compare** to show the list of commits, and changed files. To reverse + the **Source** and **Target**, select **Swap revisions**. + +## Delete merged branches + +![Delete merged branches](img/delete_merged_branches.png) + +This feature allows merged branches to be deleted in bulk. Only branches that +have been merged into the project's default branch and +[are not protected](../../protected_branches.md) are deleted as part of +this operation. + +It's particularly useful to clean up old branches that were not deleted +automatically when a merge request was merged. + +## Related topics + +- [Protected branches](../../protected_branches.md) +- [Branches API](../../../../api/branches.md) +- [Protected Branches API](../../../../api/protected_branches.md) +- [Getting started with Git](../../../../topics/git/index.md) ## Troubleshooting diff --git a/doc/user/project/repository/code_suggestions.md b/doc/user/project/repository/code_suggestions.md new file mode 100644 index 00000000000..5de55db5ad4 --- /dev/null +++ b/doc/user/project/repository/code_suggestions.md @@ -0,0 +1,182 @@ +--- +stage: ModelOps +group: AI Assisted +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference +--- + +# Code Suggestions (Beta) **(FREE SAAS)** + +> - [Introduced](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#code-suggestions-available-in-closed-beta) in GitLab 15.9 as [Beta](/ee/policy/alpha-beta-support.md#beta) for early access Ultimate customers. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/408104) as opt-in with GitLab 15.11 as [Beta](/ee/policy/alpha-beta-support.md#beta). +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/408158) from GitLab Ultimate to GitLab Premium in 16.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/410801) from GitLab Premium to GitLab Free in 16.0. + +WARNING: +This feature is in [Beta](/ee/policy/alpha-beta-support.md#beta). +Code Suggestions use generative AI to suggest code while you're developing. +Due to high demand, this feature will have unscheduled downtime and code suggestions in VS Code may be delayed. +Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). +Beta users should read about the [known limitations](#known-limitations). We look forward to hearing your feedback. + +Use Code Suggestions to write code more efficiently by viewing code suggestions +as you type. Depending on the cursor position, the extension either: + +- Provides entire code snippets, like generating functions. +- Completes the current line. + +To accept a suggestion, press <kbd>Tab</kbd>. + +Code Suggestions are available in Visual Studio Code when you have the GitLab Workflow extension installed. + +## Supported languages + +Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). The best results from Code Suggestions are expected for these languages: + +- C/C++ +- C# +- Go +- Java +- JavaScript +- Python +- PHP +- Ruby +- Rust +- Scala +- TypeScript + +Suggestions may be mixed for other languages. Using natural language code comments to request completions may also not function as expected. + +GitLab is continuously improving the model and expects to support an additional seven languages soon, as well as natural language code comments. + +Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). + +## Enable Code Suggestions in VS Code + +Prerequisites: + +- Your group owner must enable the [group level code suggestions setting](../../group/manage.md#group-code-suggestions). +- You must have [a personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` and `read_user` scopes. + +To enable Code Suggestions in VS Code: + +1. Download and configure the + [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) + for Visual Studio Code. +1. In **GitLab: Add Account to VS Code on Mac**, add your GitLab work account to the VS Code extension: + - In macOS, press <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>P</kbd>. + - In Windows, press <kbd>Shift</kbd> + <kbd>Control</kbd> + <kbd>P</kbd>. +1. Provide your GitLab instance URL. A default is provided. +1. Provide your personal access token. +1. After your GitLab account connects successfully, in the left sidebar, select **Extensions**. +1. Find the **GitLab workflow** extension, select **Settings** (**{settings}**), and select **Extension Settings**. +1. Enable **GitLab > AI Assisted Code Suggestions**. + +Start typing and receive suggestions for your GitLab projects. + +<div class="video-fallback"> + See an end-to-end demo: <a href="https://www.youtube.com/watch?v=WnxBYxN2-p4">Get started with GitLab Code Suggestions in VS Code</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe> +</figure> + +## Code Suggestions data usage + +Code Suggestions is a generative artificial intelligence (AI) model hosted on GitLab.com. + +Your personal access token enables a secure API connection to GitLab.com. +This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, +and the generated suggestion is transmitted back to VS Code. + +### Data privacy + +Code Suggestions operate completely in the GitLab.com infrastructure, providing the same level of +[security](https://about.gitlab.com/security/) as any other feature of GitLab.com, and processing any personal +data in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). + +No new additional data is collected to enable this feature. The content of your GitLab hosted source code is +not used as training data. Source code inference against the Code Suggestions model is not used to re-train the model. +Your data also never leaves GitLab.com. All training and inference is done in GitLab.com infrastructure. + +[Read more about the security of GitLab.com](https://about.gitlab.com/security/faq/). + +### Training data + +Code Suggestions uses open source pre-trained base models from the +[CodeGen family](https://openreview.net/forum?id=iaYcJKpY2B_) including CodeGen-MULTI and CodeGen-NL. +We then re-train and fine-tune these base models with a customized open source dataset to enable multi-language +support and additional use cases. This customized dataset contains non-preprocessed open source code in 13 +programming languages from [The Pile](https://pile.eleuther.ai/) and the +[Google BigQuery source code dataset](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). +We then process this raw dataset against heuristics that aim to increase the quality of the dataset. + +The Code Suggestions model is not trained on GitLab customer data. + +### Off by default + +Code Suggestions are off by default and require a group owner to enable the feature with a group-level setting. + +After the group-level setting is enabled, developers using Visual Studio Code with the +[GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) can connect +to GitLab.com by using a GitLab +[personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) with the `read_api` +and `read_user` scopes. + +## Progressive enhancement + +This feature is designed as a progressive enhancement to the existing VS Code GitLab Workflow plugin. +Code Suggestions offer a completion if the machine learning engine can generate a recommendation. +In the event of a connection issue or model inference failure, the feature gracefully degrades. +Code Suggestions do not prevent you from writing code in VS Code. + +### Internet connectivity + +Code Suggestions only work when you have internet connectivity and can access GitLab.com. +Code Suggestions are not available for self-managed customers, nor customers operating within an offline environment. + +### Stability and performance + +This feature is currently in [Beta](/ee/policy/alpha-beta-support.md#beta). +While the Code Suggestions inference API operates completely within the GitLab.com enterprise infrastructure, +we expect a high demand for this Beta feature, which may cause degraded performance or unexpected downtime +of the feature. We have built this feature to gracefully degrade and have controls in place to allow us to +mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion. + +### Model accuracy and quality + +While in Beta, Code Suggestions can generate low-quality, incomplete, and possibly insecure code. +We strongly encourage all beta users to leverage GitLab native +[Code Quality Scanning](../../../ci/testing/code_quality.md) and +[Security Scanning](../../application_security/index.md) capabilities. + +GitLab uses a customized open source dataset to fine-tune the model to support multiple languages. +Based on the languages you code in, GitLab routes the request to a targeted inference and prompt engine +to get relevant suggestions. + +GitLab is actively refining these models to: + +- Improve the quality of recommendations. +- Add support for more languages. +- Add protections to limit personal data, insecure code, and other unwanted behavior + that the model may have learned from training data. + +## Known limitations + +While in Beta, we are working on improving the accuracy of overall generated content. +However, Code Suggestions may generate suggestions that are: + +- Low-quality +- Incomplete +- Produce failed pipelines +- Insecure code +- Offensive or insensitive + +We are also aware of specific situations that can produce unexpected or incoherent results including: + +- Suggestions based on natural language code comments. +- Suggestions that mixed programming languages in unexpected ways. + +## Feedback + +Report issues in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/405152). diff --git a/doc/user/project/repository/file_finder.md b/doc/user/project/repository/file_finder.md index f9a5d4e3aa7..e22dc549a4a 100644 --- a/doc/user/project/repository/file_finder.md +++ b/doc/user/project/repository/file_finder.md @@ -1,42 +1,28 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/file_finder.html' --- # File finder **(FREE)** -The file finder feature allows you to search for a file in a repository using the -GitLab UI. To use it: +With file finder, you can search for a file in a repository directly from the GitLab UI. -1. Go to your project's **Repository > Files**. -1. In the upper right corner, select **Find File**. +File finder is powered by the [`fuzzaldrin-plus`](https://github.com/jeancroy/fuzz-aldrin-plus) library, which uses fuzzy search and highlights results as you type. -If you prefer to keep your fingers on the keyboard, use the -[shortcut button](../../shortcuts.md), which you can invoke from anywhere -in a project. +## Search for a file -Press `t` to launch the File search function when in **Issues**, -**Merge requests**, **Milestones**, even the project's settings. +To search for a file: -Start typing what you are searching for and watch the magic happen. With the -up/down arrows, you go up and down the results, with `Esc` you close the search -and go back to **Files** +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files**. +1. In the upper right, select **Find file**. +1. In the search box, start typing the filename. +1. From the dropdown list, select the file. -## How it works +To go to file finder, you can also press <kbd>t</kbd> anywhere in your project. +To go back to **Files**, press <kbd>Esc</kbd>. -The File finder feature is powered by the [Fuzzy filter](https://github.com/jeancroy/fuzz-aldrin-plus) library. - -It implements a fuzzy search with the highlight and tries to provide intuitive -results by recognizing patterns that people use while searching. - -For example, consider the [GitLab FOSS repository](https://gitlab.com/gitlab-org/gitlab-foss/tree/master) and that we want to open -the `app/controllers/admin/deploy_keys_controller.rb` file. - -Using a fuzzy search, we start by typing letters that get us closer to the file. - -NOTE: -To narrow down your search, include `/` in your search terms. +To narrow down your results, include `/` in your search. ![Find file button](img/file_finder_find_file_v12_10.png) diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index 929751b7247..74d765fa7fe 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/forking_workflow.html' --- # Project forking workflow **(FREE)** @@ -16,14 +15,14 @@ A fork is a personal copy of the repository and all its branches, which you crea in a namespace of your choice. Make changes in your own fork and submit them through a merge request to the repository you don't have access to. -## Creating a fork +## Create a fork > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) a new form in GitLab 13.11 [with a flag](../../../user/feature_flags.md) named `fork_project_form`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77181) in GitLab 14.8. Feature flag `fork_project_form` removed. To fork an existing project in GitLab: -1. On the project's home page, in the upper right, select **{fork}** **Fork**: +1. On the project's homepage, in the upper-right corner, select **Fork** (**{fork}**): ![Fork this project](img/forking_workflow_fork_button_v13_10.png) 1. Optional. Edit the **Project name**. 1. For **Project URL**, select the [namespace](../../namespace/index.md) @@ -39,10 +38,51 @@ GitLab creates your fork, and redirects you to the new fork's page. ## Update your fork -To copy the latest changes from the upstream repository into your fork, update it -[from the command line](#from-the-command-line). GitLab Premium and higher tiers can also -[configure forks as pull mirrors](#with-repository-mirroring) -of the upstream repository. +A fork can fall out of sync with its upstream repository, and require an update: + +- **Ahead**: Your fork contains new commits not present in the upstream repository. + To sync your fork, create a merge request to push your changes to the upstream repository. +- **Behind**: The upstream repository contains new commits not present in your fork. + To sync your fork, pull the new commits into your fork. +- **Ahead and behind**: Both the upstream repository and your fork contain new commits + not present in the other. To fully sync your fork, create a merge request to push + your changes up, and pull the upstream repository's new changes into your fork. + +To sync your fork with its upstream repository, update it from the GitLab UI +or the command line. GitLab Premium and Ultimate tiers can also automate updates by +[configuring forks as pull mirrors](#with-repository-mirroring) of the upstream repository. + +### From the UI + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330243) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `synchronize_fork`. Disabled by default, but enabled for projects in the `gitlab-org/gitlab` and `gitlab-com/www-gitlab-com` namespaces only. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `synchronize_fork`. +On GitLab.com, this feature is available for projects in the `gitlab-org/gitlab` and `gitlab-com/www-gitlab-com` namespaces. + +To update your fork from the GitLab UI: + +1. On the top bar, select **Main menu > Projects > View all projects**. +1. On the secondary menu, select **Personal**. +1. Select the fork you want to update. +1. Below the dropdown list for branch name, find the **Forked from** (**{fork}**) + information box to determine if your fork is ahead, behind, or both. In this example, + the fork is behind the upstream repository: + + ![Information box for a fork 23552 commits behind the upstream repository](img/update-fork_v15_11.png) + +1. If your fork is **ahead** of the upstream repository, select + **Create merge request** to propose adding your fork's changes to the upstream repository. +1. If your fork is **behind** the upstream repository, select **Update fork** + to pull changes from the upstream repository. +1. If your fork is **ahead and behind** the upstream repository, you can update from the UI + available only if no merge conflicts are detected: + - If your fork contains no merge conflicts, you can select **Create merge request** + to propose pushing your changes to the upstream repository, **Update fork** + to pull changes down to your fork, or both. The type of changes in your fork + determine which actions are appropriate. + - If your fork contains merge conflicts, update your fork from the command line. ### From the command line @@ -102,7 +142,7 @@ an `upstream` remote repository for your fork: A fork can be configured as a mirror of the upstream if all these conditions are met: -1. Your subscription is **(PREMIUM)** or a higher tier. +1. Your subscription is **Premium** or **Ultimate**. 1. You create all changes in branches (not `main`). 1. You do not work on [merge requests for confidential issues](../merge_requests/confidential.md), which requires changes to `main`. @@ -114,7 +154,7 @@ For instructions, read [Configure pull mirroring](mirror/pull.md#configure-pull- WARNING: With mirroring, before approving a merge request, you are asked to sync. You should automate it. -## Merging upstream +## Merge changes back upstream When you are ready to send your code back to the upstream project, [create a merge request](../merge_requests/creating_merge_requests.md). For **Source branch**, @@ -129,11 +169,55 @@ Then you can add labels, a milestone, and assign the merge request to someone wh your changes. Then select **Submit merge request** to conclude the process. When successfully merged, your changes are added to the repository and branch you're merging into. -## Removing a fork relationship +## Unlink a fork -You can unlink your fork from its upstream project in the [advanced settings](../settings/index.md#remove-a-fork-relationship). +Removing a fork relationship unlinks your fork from its upstream project. +Your fork then becomes an independent project. + +Prerequisites: + +- You must be a project owner to unlink a fork. + +WARNING: +If you remove a fork relationship, you can't send merge requests to the source. +If anyone has forked your project, their fork also loses the relationship. +To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). + +To remove a fork relationship: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. In the **Remove fork relationship** section, select **Remove fork relationship**. +1. To confirm, enter the project path and select **Confirm**. + +When you unlink a fork that uses a [hashed storage pool](../../../administration/repository_storage_types.md#hashed-object-pools) +to share objects with another repository: + +- All objects are copied from the pool into your fork. +- After the copy process completes, no further updates from the storage pool are propagated to your fork. ## Related topics -- GitLab blog post: [How to keep your fork up to date with its origin](https://about.gitlab.com/blog/2016/12/01/how-to-keep-your-fork-up-to-date-with-its-origin/). -- GitLab community forum: [Refreshing a fork](https://forum.gitlab.com/t/refreshing-a-fork/). +- GitLab blog post: [Keep your fork up to date with its origin](https://about.gitlab.com/blog/2016/12/01/how-to-keep-your-fork-up-to-date-with-its-origin/) +- GitLab community forum: [Refreshing a fork](https://forum.gitlab.com/t/refreshing-a-fork/32469) + +## Troubleshooting + +### Error: `An error occurred while forking the project. Please try again` + +This error can be due to a mismatch in shared runner settings between the forked project +and the new namespace. See [Forks](../../../ci/runners/configure_runners.md#forks) +in the Runner documentation for more information. + +### Removing fork relationship fails + +If removing the fork through the UI or API is not working, you can attempt the +fork relationship removal in a +[Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + +```ruby +p = Project.find_by_full_path('<project_path>') +u = User.find_by_username('<username>') +Projects::UnlinkForkService.new(p, u).execute +``` diff --git a/doc/user/project/repository/git_blame.md b/doc/user/project/repository/git_blame.md index 78fcdec2a0b..fbf29bddd46 100644 --- a/doc/user/project/repository/git_blame.md +++ b/doc/user/project/repository/git_blame.md @@ -14,7 +14,7 @@ commit hash. To view it for a file: 1. Go to your project's **Repository > Files**. 1. Select the file you want to review. -1. In the upper right corner, select **Blame**. +1. In the upper-right corner, select **Blame**. When you select **Blame**, this information is displayed: diff --git a/doc/user/project/repository/git_history.md b/doc/user/project/repository/git_history.md index 321b9a5eb53..a9228b8cabd 100644 --- a/doc/user/project/repository/git_history.md +++ b/doc/user/project/repository/git_history.md @@ -12,7 +12,7 @@ Git file History provides information about the commit history associated with a file. To use it: 1. Go to your project's **Repository > Files**. -1. In the upper right corner, select **History**. +1. In the upper-right corner, select **History**. When you select **History**, this information is displayed: diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 64f19d1a92c..a2dd2488961 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -43,7 +43,7 @@ To view a user's public GPG key, you can either: - Go to `https://gitlab.example.com/<USERNAME>.gpg`. GitLab displays the GPG key, if the user has configured one, or a blank page for users without a configured GPG key. -- Go to the user's profile (such as `https://gitlab.example.com/<USERNAME>`). In the upper right +- Go to the user's profile (such as `https://gitlab.example.com/<USERNAME>`). In the upper-right corner of the user's profile, select **View public GPG keys** (**{key}**). ## Configure commit signing diff --git a/doc/user/project/repository/img/update-fork_v15_11.png b/doc/user/project/repository/img/update-fork_v15_11.png Binary files differnew file mode 100644 index 00000000000..244868d80ae --- /dev/null +++ b/doc/user/project/repository/img/update-fork_v15_11.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 5b4e1b14489..9f0c46591b9 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -25,7 +25,7 @@ You can add files to a repository: - When you create a project. - After you create a project: - By using [the web editor](web_editor.md). - - [From the command line](../../../gitlab-basics/command-line-commands.md). + - From the command line. ## Commit changes to a repository @@ -235,9 +235,9 @@ The size can differ slightly from one instance to another due to compression, ho Administrators can set a [repository size limit](../../admin_area/settings/account_and_limit_settings.md). [GitLab sets the size limits for GitLab.com](../../gitlab_com/index.md#account-and-limit-settings). -## Repository contributor graph +## Repository contributor statistics -All code contributors are displayed under your project's **Repository > Contributors**. +All code contributors are displayed under your project's **Repository > Contributor statistics**. The graph shows the contributor with the most commits to the fewest. @@ -277,15 +277,15 @@ to fetch configuration from a project that is renamed or moved. ## Related topics -- [GitLab Workflow VS Code extension](vscode.md). -- To lock files and prevent change conflicts, use [file locking](../file_lock.md). -- [Repository API](../../../api/repositories.md). -- [Find files](file_finder.md) in a repository. -- [Branches](branches/index.md). -- [Create a directory](web_editor.md#create-a-directory). -- [Find file history](git_history.md). -- [Identify changes by line (Git blame)](git_blame.md). -- [Use Jupyter notebooks with GitLab](jupyter_notebooks/index.md). +- [GitLab Workflow VS Code extension](vscode.md) +- [Lock files and prevent change conflicts](../file_lock.md) +- [Repository API](../../../api/repositories.md) +- [Find files](file_finder.md) +- [Branches](branches/index.md) +- [Create a directory](web_editor.md#create-a-directory) +- [Find file history](git_history.md) +- [Identify changes by line (Git blame)](git_blame.md) +- [Use Jupyter notebooks with GitLab](jupyter_notebooks/index.md) ## Troubleshooting diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md index 28f0ffb6fbd..1526c7b4dc2 100644 --- a/doc/user/project/repository/jupyter_notebooks/index.md +++ b/doc/user/project/repository/jupyter_notebooks/index.md @@ -25,7 +25,7 @@ GitLab. ## Cleaner diffs and raw diffs -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Experiment](../../../../policy/alpha-beta-support.md#experiment) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed. > - [Reintroduced toggle](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85079) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95373) in GitLab 15.6. Feature flag `ipynb_semantic_diff` removed. diff --git a/doc/user/project/repository/mirror/bidirectional.md b/doc/user/project/repository/mirror/bidirectional.md index 9942469dd5c..0563f747e8d 100644 --- a/doc/user/project/repository/mirror/bidirectional.md +++ b/doc/user/project/repository/mirror/bidirectional.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Bidirectional mirroring **(PREMIUM)** @@ -168,4 +167,4 @@ Read about [Git Fusion settings on Perforce.com](https://www.perforce.com/manual ## Related topics -- [Configure server hooks](../../../../administration/server_hooks.md) on a GitLab server. +- [Configure server hooks](../../../../administration/server_hooks.md) diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index 7a50c294dfc..9f72b8f29b2 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Repository mirroring **(FREE)** @@ -107,13 +106,14 @@ To use this option, select **Only mirror protected branches** when you create a ## Mirror specific branches -> Mirroring branches matching a regex [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102608) in GitLab 15.8 [with a flag](../../../../administration/feature_flags.md) named `mirror_only_branches_match_regex`. Disabled by default. +> - Mirroring branches matching a regex [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102608) in GitLab 15.8 [with a flag](../../../../administration/feature_flags.md) named `mirror_only_branches_match_regex`. Disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 16.0. FLAG: -On self-managed GitLab, by default the field `mirror_branch_regex` is not available. -To make it available, ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) +On self-managed GitLab, by default the field `mirror_branch_regex` is available. +To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `mirror_only_branches_match_regex`. -On GitLab.com, this feature is not available. +On GitLab.com, this feature is available. To mirror only branches with names matching an [re2 regular expression](https://github.com/google/re2/wiki/Syntax), enter a regular expression into the **Mirror specific branches** field. Branches with names that @@ -331,6 +331,29 @@ and requires the full version including the protocol (`ssh://git@gitlab.com/gitl Make sure that host and project path are separated using `/` instead of `:`. +### Host key verification failed + +This error is returned when the target host public SSH key changes. +Public SSH keys rarely, if ever, change. If host key verification fails, +but you suspect the key is still valid, you can refresh the key's information. + +Prerequisites: + +- You must have at least the Maintainer role for a project. + +To resolve the issue: + +1. [Verify the host key](#verify-a-host-key). +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Mirroring repositories**. +1. To refresh the keys, either: + + - Select **Detect host keys** for GitLab to fetch the host keys from the server, and display the fingerprints. + - Select **Input host keys manually**, and enter the host key into the **SSH host key** field. + +- Select **Mirror repository**. + ### Transfer mirror users and tokens to a single service account in Rails console This requires access to the [GitLab Rails console](../../../../administration/operations/rails_console.md#starting-a-rails-console-session). diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index 0eb51f40208..2b8470b9e3d 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Pull from a remote repository **(PREMIUM)** @@ -142,6 +141,5 @@ end ## Related topics -- Configure [pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) - on self-managed instances. -- Configure [pull mirroring through the API](../../../../api/projects.md#configure-pull-mirroring-for-a-project). +- [Pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) +- [Pull mirroring API](../../../../api/projects.md#configure-pull-mirroring-for-a-project) diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index f06dd747897..11093958650 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -2,7 +2,6 @@ stage: Create group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' --- # Push mirroring **(FREE)** @@ -26,8 +25,10 @@ When you push a change to the upstream repository, the push mirror receives it: - Within five minutes. - Within one minute, if you enabled **Only mirror protected branches**. -In the case of a diverged branch, an error displays in the **Mirroring repositories** -section. +When a branch is merged into the default branch and deleted in the source project, +it is deleted from the remote mirror on the next push. Branches with unmerged +changes are kept. If a branch diverges, the **Mirroring repositories** section +displays an error. ## Configure push mirroring @@ -194,4 +195,4 @@ If it isn't working correctly, a red `error` tag appears, and shows the error me ## Related topics -- [Remote mirrors API](../../../../api/remote_mirrors.md). +- [Remote mirrors API](../../../../api/remote_mirrors.md) diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md index 7ae085812ae..28afba375fc 100644 --- a/doc/user/project/repository/push_rules.md +++ b/doc/user/project/repository/push_rules.md @@ -13,7 +13,7 @@ can and can't be pushed to your repository. While GitLab offers - Evaluating the contents of a commit. - Confirming commit messages match expected formats. -- Enforcing branch name rules. +- Enforcing [branch name rules](branches/index.md#name-your-branch). - Evaluating the details of files. - Preventing Git tag removal. diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index cf7d4f44aad..4bebe4c1a97 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -22,16 +22,27 @@ Rewriting repository history is a destructive operation. Make sure to back up yo you begin. The best way to back up a repository is to [export the project](../settings/import_export.md#export-a-project-and-its-data). +## Calculate repository size + +The size of a repository is determined by computing the accumulated size of all files in the repository. +It is similar to executing `du --summarize --bytes` on your repository's +[hashed storage path](../../../administration/repository_storage_types.md). + ## Purge files from repository history -To reduce the size of your repository in GitLab, you must first remove references to large files from branches, tags, *and* -other internal references (refs) that are automatically created by GitLab. These refs include: +GitLab [prunes unreachable objects](../../../administration/housekeeping.md#prune-unreachable-objects) +as part of housekeeping. In GitLab, to reduce the disk size of your repository manually, you must +first remove references to large files from branches, tags, *and* other internal references (refs) +created by GitLab. These refs include: -- `refs/merge-requests/*` for merge requests. -- `refs/pipelines/*` for - [pipelines](../../../ci/troubleshooting.md#fatal-reference-is-not-a-tree-error). -- `refs/environments/*` for environments. -- `refs/keep-around/*` are created as hidden refs to prevent commits referenced in the database from being removed +- `refs/merge-requests/*` +- `refs/pipelines/*` +- `refs/environments/*` +- `refs/keep-around/*` + +NOTE: +For details on each of these references, see +[GitLab-specific references](../../../development/gitaly.md#gitlab-specific-references). These refs are not automatically downloaded and hidden refs are not advertised, but we can remove these refs using a project export. diff --git a/doc/user/project/repository/tags/img/tag-display_v15_9.png b/doc/user/project/repository/tags/img/tag-display_v15_9.png Binary files differnew file mode 100644 index 00000000000..015df07d025 --- /dev/null +++ b/doc/user/project/repository/tags/img/tag-display_v15_9.png diff --git a/doc/user/project/repository/tags/img/tags_commits_view_v15_10.png b/doc/user/project/repository/tags/img/tags_commits_view_v15_10.png Binary files differnew file mode 100644 index 00000000000..247d2f368d5 --- /dev/null +++ b/doc/user/project/repository/tags/img/tags_commits_view_v15_10.png diff --git a/doc/user/project/repository/tags/index.md b/doc/user/project/repository/tags/index.md new file mode 100644 index 00000000000..e252a9a433b --- /dev/null +++ b/doc/user/project/repository/tags/index.md @@ -0,0 +1,120 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Tags **(FREE)** + +In Git, a tag marks an important point in a repository's history. +Git supports two types of tags: + +- **Lightweight tags** point to specific commits, and contain no other information. + Also known as soft tags. Create or remove them as needed. +- **Annotated tags** contain metadata, can be signed for verification purposes, + and can't be changed. + +The creation or deletion of a tag can be used as a trigger for automation, including: + +- Using a [webhook](../../integrations/webhook_events.md#tag-events) to automate actions + like Slack notifications. +- Signaling a [repository mirror](../mirror/index.md) to update. +- Running a CI/CD pipeline with [`if: $CI_COMMIT_TAG`](../../../../ci/jobs/job_control.md#common-if-clauses-for-rules). + +When you [create a release](../../releases/index.md), +GitLab also creates a tag to mark the release point. +Many projects combine an annotated release tag with a stable branch. Consider +setting deployment or release tags automatically. + +In the GitLab UI, each tag displays: + +![Example of a single tag](img/tag-display_v15_9.png) + +- The tag name. (**{tag}**) +- Optional. If the tag is [protected](../../protected_tags.md), a **protected** badge. +- The commit SHA (**{commit}**), linked to the commit's contents. +- The commit's title and creation date. +- Optional. A link to the release (**{rocket}**). +- Optional. If a pipeline has been run, the current pipeline status. +- Download links to the source code and artifacts linked to the tag. +- A [**Create release**](../../releases/index.md#create-a-release) (**{pencil}**) link. +- A link to delete the tag. + +## View tags for a project + +To view all existing tags for a project: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Tags**. + +## View tagged commits in the commits list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18795) in GitLab 15.10. + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Commits**. +1. Commits with a tag are labeled with a tag icon (**{tag}**) and the name of the tag. + This example shows a commit tagged `v1.26.0`: + + ![A tagged commit in the Commits view](img/tags_commits_view_v15_10.png) + +To view the list of commits in this tag, select the tag name. + +## Create a tag + +Tags can be created from the command line, or the GitLab UI. + +### From the command line + +To create either a lightweight or annotated tag from the command line, and push it upstream: + +1. To create a lightweight tag, run the command `git tag TAG_NAME`, changing + `TAG_NAME` to your desired tag name. +1. To create an annotated tag, run one of the versions of `git tag` from the command line: + + ```shell + # In this short version, the annotated tag's name is "v1.0", + # and the message is "Version 1.0". + git tag -a v1.0 -m "Version 1.0" + + # Use this version to write a longer tag message + # for annotated tag "v1.0" in your text editor. + git tag -a v1.0 + ``` + +1. Push your tags upstream with `git push origin --tags`. + +### From the UI + +To create a tag from the GitLab UI: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Tags**. +1. Select **New tag**. +1. Provide a **Tag name**. +1. For **Create from**, select an existing branch name, tag, or commit SHA. +1. Optional. Add a **Message** to create an annotated tag, or leave blank to + create a lightweight tag. +1. Select **Create tag**. + +## Prevent tag deletion **(PREMIUM)** + +To prevent users from removing a tag with `git push`, create a [push rule](../push_rules.md). + +## Trigger pipelines from a tag + +GitLab CI/CD provides a [`CI_COMMIT_TAG` variable](../../../../ci/variables/predefined_variables.md) +to identify tags. Use this variable in job rules and workflow rules to test if the pipeline +was triggered by a tag. + +In your `.gitlab-ci.yml` file for the CI/CD pipeline configuration of your project, +you can trigger based on a new tag: + +- At the job level, with the [`only` keyword](../../../../ci/yaml/index.md#only--except). +- At the pipeline level, with the [workflow rules keywords](../../../../ci/yaml/workflow.md). + +## Related topics + +- [Tagging (Git reference page)](https://git-scm.com/book/en/v2/Git-Basics-Tagging) +- [Protected tags](../../protected_tags.md) +- [Tags API](../../../../api/tags.md) diff --git a/doc/user/project/repository/vscode.md b/doc/user/project/repository/vscode.md index 9260d59bc3a..2a33476b545 100644 --- a/doc/user/project/repository/vscode.md +++ b/doc/user/project/repository/vscode.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -22,6 +22,7 @@ do more day-to-day tasks in Visual Studio Code, such as: and paste snippets to, and from, your editor. - [Browse repositories](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#browse-a-repository-without-cloning) without cloning them. +- [Receive Code Suggestions](code_suggestions.md) ## Download the extension @@ -34,6 +35,7 @@ you can [configure](https://marketplace.visualstudio.com/items?itemName=GitLab.g - [Features to display or hide](https://gitlab.com/gitlab-org/gitlab-vscode-extension#extension-settings). - [Self-signed certificate](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#self-signed-certificates) information. +- [Code Suggestions](code_suggestions.md) ## Report issues with the extension @@ -44,5 +46,4 @@ Report any issues, bugs, or feature requests in the - [Download the extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) - [Extension documentation](https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob/main/README.md) -- The extension source code is available in the - [`gitlab-vscode-extension`](https://gitlab.com/gitlab-org/gitlab-vscode-extension/) project. +- [View source code](https://gitlab.com/gitlab-org/gitlab-vscode-extension/) diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 80b0ada6f7b..dc988846676 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -1,6 +1,6 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -26,11 +26,32 @@ for any change you commit through the Web Editor. To create a text file in the Web Editor: 1. On the top bar, select **Main menu > Projects** and find your project. -1. From the project dashboard or repository, next to the branch name, select the plus icon (**{plus}**). +1. From the project dashboard or repository, next to the branch name, + select the plus icon (**{plus}**). 1. From the dropdown list, select **New file**. -1. Complete the fields. To create a merge request with the new file, ensure the **Start a new merge request with these changes** checkbox is selected. +1. Complete the fields. +1. To create a merge request with the new file, ensure the **Start a new merge request with these changes** checkbox is selected, if you had chosen a **Target branch** other than the [default branch (such as `main`)](../../../user/project/repository/branches/default.md). 1. Select **Commit changes**. +### Create a file from a template + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Files**. +1. Next to the project name, select the plus icon (**{plus}**) to display a + dropdown list, then select **New file** from the list. +1. For **Filename**, provide one of the filenames that GitLab provides a template for: + - `.gitignore` + - `.gitlab-ci.yml` + - `LICENSE` + - `Dockerfile` +1. Select **Apply a template**, then select the template you want to apply. +1. Make your changes to the file. +1. Provide a **Commit message**. +1. Enter a **Target branch** to merge into. To create a new merge request with + your changes, enter a branch name that is not your repository's + [default branch](../../../user/project/repository/branches/default.md), +1. Select **Commit changes** to add the commit to your branch. + ## Edit a file To edit a text file in the Web Editor: @@ -87,6 +108,9 @@ To link to a single line, you can also: To upload a binary file in the Web Editor: +<!-- This list is duplicated at doc/gitlab-basics/add-file.md#from-the-ui --> +<!-- For why we duplicated the info, see https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111072#note_1267429478 --> + 1. On the top bar, select **Main menu > Projects** and find your project. 1. From the project dashboard or repository, next to the branch name, select the plus icon (**{plus}**). 1. From the dropdown list, select **Upload file**. @@ -115,7 +139,7 @@ To create a [branch](branches/index.md) in the Web Editor: ## Create a tag -You can create [tags](../../../topics/git/tags.md) to mark milestones such as +You can create [tags](tags/index.md) to mark milestones such as production releases and release candidates. To create a tag in the Web Editor: 1. On the top bar, select **Main menu > Projects** and find your project. diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index 42f7be30822..80538697100 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -32,7 +32,7 @@ For a commit or tag to be *verified* by GitLab: from the certificate in the signature to a trusted certificate in the GitLab certificate store. This chain may include intermediate certificates supplied in the signature. You may need to add certificates, such as Certificate Authority root certificates, - [to the GitLab certificate store](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). + [to the GitLab certificate store](https://docs.gitlab.com/omnibus/settings/ssl/index.html#install-custom-public-certificates). - The signing time must be in the time range of the [certificate validity](https://www.rfc-editor.org/rfc/rfc5280.html#section-4.1.2.5), which is usually up to three years. @@ -101,7 +101,7 @@ To configure Windows or macOS: - Downloading the installer. - Running `brew install smimesign` on macOS. 1. Get the ID of your certificate by running `smimesign --list-keys`. -1. Set your signing key by running `git config --global user.signingkey ID`. +1. Set your signing key by running `git config --global user.signingkey <ID>`, replacing `<ID>` with the certificate ID. 1. Configure X.509 with this command: ```shell @@ -342,7 +342,7 @@ step of the previous [main verification checks](#main-verification-checks). ``` 1. If this fails, add the missing certificates required to establish trust - [to the GitLab certificate store](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). + [to the GitLab certificate store](https://docs.gitlab.com/omnibus/settings/ssl/index.html#install-custom-public-certificates). 1. After adding more certificates, (if these troubleshooting steps then pass) run the Rake task to [re-verify commits](#re-verify-commits). diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index b04905e6b34..b8e9de41062 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -1,7 +1,7 @@ --- type: reference, howto stage: Plan -group: Certify +group: Product Planning info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -24,12 +24,17 @@ If an industry standard *requires* that your application has a certain feature o [create a requirement](#create-a-requirement) to reflect this. When a feature is no longer necessary, you can [archive the related requirement](#archive-a-requirement). +NOTE: +Requirements and [test cases](../../../ci/test_cases/index.md) are being +[migrated to work items](https://gitlab.com/groups/gitlab-org/-/epics/5171). +[Issue 323790](https://gitlab.com/gitlab-org/gitlab/-/issues/323790) proposes to link requirements to test cases. +For more information, see [Product Stage Direction - Plan](https://about.gitlab.com/direction/plan/). + <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [GitLab 12.10 Introduces Requirements Management](https://www.youtube.com/watch?v=uSS7oUNSEoU). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a more in-depth walkthrough using a [demonstration project](https://gitlab.com/gitlab-org/requiremeents-mgmt), -see [GitLab Requirements Traceability Walkthrough](https://youtu.be/VIiuTQYFVa0) (Feb 2021). +For a more in-depth walkthrough see [GitLab Requirements Traceability Walkthrough](https://youtu.be/VIiuTQYFVa0) (Feb 2021). ![requirements list view](img/requirements_list_v13_5.png) @@ -235,9 +240,9 @@ Before you import your file: To import requirements: 1. In a project, go to **Issues > Requirements**. - - If the project already has existing requirements, select the import icon (**{import}**) in the - upper right. - - For a project without any requirements, select **Import CSV** in the middle of the page. + - For a project with requirements, in the + upper-right corner, select the import icon (**{import}**). + - For a project without requirements, in the middle of the page, select **Import CSV**. 1. Select the file and select **Import requirements**. The file is processed in the background and a notification email is sent @@ -296,7 +301,7 @@ Prerequisite: To export requirements: 1. In a project, go to **Issues > Requirements**. -1. In the upper right, select the **Export as CSV** icon (**{export}**). +1. In the upper-right corner, select **Export as CSV** (**{export}**). A confirmation modal appears. diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index 22297149561..bd5c8214e68 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -8,29 +8,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Moved to GitLab Free in 13.2. -Service Desk is a module that allows your team to connect -with any external party through email, without any external tools. -An ongoing conversation in the same place as where your software -is built ensures user feedback ends up where it's needed. +With Service Desk, your customers +can email you bug reports, feature requests, or general feedback. +Service Desk provides a unique email address, so they don't need their own GitLab accounts. -With Service Desk, you can provide efficient email support to your customers. They can -email you bug reports, feature requests, or general feedback. They all end up in your -GitLab project as new issues. In turn, your team can respond directly from the project. +Service Desk emails are created in your GitLab project as new issues. +Your team can respond directly from the project, while customers interact with the thread only +through email. -As Service Desk is built right into GitLab itself, the complexity and inefficiencies -of multiple tools and external integrations are eliminated. This significantly shortens -the cycle time from feedback to software update. - -For an overview, check the video demonstration on [GitLab Service Desk](https://about.gitlab.com/blog/2017/05/09/demo-service-desk/). - -## How it works - -GitLab Service Desk enables people to create issues in your -GitLab instance without needing their own user account. - -It provides a unique email address for end users to create issues in a project. -Follow-up notes can be sent either through the GitLab interface or by email. End -users only see the thread through email. +## Service Desk workflow For example, let's assume you develop a game for iOS or Android. The codebase is hosted in your GitLab instance, built and deployed @@ -43,57 +29,58 @@ Here's how Service Desk works for you: 1. Each email they send creates an issue in the appropriate project. 1. Your team members go to the Service Desk issue tracker, where they can see new support requests and respond inside associated issues. -1. Your team communicates back and forth with the customer to understand the request. +1. Your team communicates with the customer to understand the request. 1. Your team starts working on implementing code to solve your customer's problem. -1. When your team finishes the implementation, whereupon the merge request is merged and the issue +1. When your team finishes the implementation, the merge request is merged and the issue is closed automatically. -1. The customer's requests are handled through email, without ever having access to your - GitLab instance. -1. Your team saved time by not having to leave GitLab (or setup any integrations) to follow up with - your customer. -## Configuring Service Desk +Meanwhile: -Users with Maintainer and higher access in a project can configure Service Desk. +- The customer interacts with your team entirely through email, without needing access to your + GitLab instance. +- Your team saves time by not having to leave GitLab (or set up integrations) to follow up with + your customer. -Service Desk issues are [confidential](issues/confidential_issues.md), so they are -only visible to project members. In GitLab 11.7 we updated the generated email -address format. The older format is still supported, so existing aliases or -contacts still work. +## Configure Service Desk -If you have [templates](description_templates.md) in your repository, you can optionally select -one from the selector menu to append it to all Service Desk issues. +To start using Service Desk for a project, you must first turn it on. +By default, Service Desk is turned off. + +Prerequisites: + +- You must have at least the Maintainer role for the project. +- On GitLab self-managed, you must [set up incoming email](../../administration/incoming_email.md#set-it-up) + for the GitLab instance. You should use + [email sub-addressing](../../administration/incoming_email.md#email-sub-addressing), + but you can also use [catch-all mailboxes](../../administration/incoming_email.md#catch-all-mailbox). + To do this, you must have administrator access. To enable Service Desk in your project: -1. (GitLab self-managed only) [Set up incoming email](../../administration/incoming_email.md#set-it-up) for the GitLab instance. - You should use [email sub-addressing](../../administration/incoming_email.md#email-sub-addressing), - but you can also use [catch-all mailboxes](../../administration/incoming_email.md#catch-all-mailbox). -1. In a project, in the left sidebar, go to **Settings > General** and expand the **Service Desk** section. -1. Enable the **Activate Service Desk** toggle. This reveals a unique email address to email issues - to the project. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Turn on the **Activate Service Desk** toggle. +1. Optional. Complete the fields. + - [Add a suffix](#configure-a-custom-email-address-suffix) to your Service Desk email address. + - If the list below **Template to append to all Service Desk issues** is empty, create a + [description template](description_templates.md) in your repository. +1. Select **Save changes**. -Service Desk is now enabled for this project! To access it in a project, in the left sidebar, select -**Issues > Service Desk**. +Service Desk is now enabled for this project. +If anyone sends an email to the address available below **Email address to use for Service Desk**, +GitLab creates a confidential issue with the email's content. -WARNING: -Anyone in your project can use the Service Desk email address to create an issue in this project, **regardless -of their access level** to your GitLab instance. +### Improve your project's security -To improve your project's security, you should: +To improve your Service Desk project's security, you should: - Put the Service Desk email address behind an alias on your email system so you can change it later. - [Enable Akismet](../../integration/akismet.md) on your GitLab instance to add spam checking to this service. Unblocked email spam can result in many spam issues being created. -The unique internal email address is visible to project members at least -the Reporter role in your GitLab instance. -An external user (issue creator) cannot see the internal email address -displayed in the information note. +### Create customized email templates -### Using customized email templates - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2460) in GitLab 12.7. > - Moved from GitLab Premium to GitLab Free in 13.2. > - `UNSUBSCRIBE_URL`, `SYSTEM_HEADER`, `SYSTEM_FOOTER`, and `ADDITIONAL_TEXT` placeholders [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285512) in GitLab 15.9. @@ -103,14 +90,6 @@ An email is sent to the author when: - A new note is created on a Service Desk issue. You can customize the body of these email messages with templates. -Save your templates in the `.gitlab/service_desk_templates/` -directory in your repository. - -With Service Desk, you can use templates for: - -- [Thank you emails](#thank-you-email) -- [New note emails](#new-note-email) -- [New Service Desk issues](#new-service-desk-issues) #### Email header and footer **(FREE SELF)** @@ -122,13 +101,18 @@ visible in the email template. For more information, see #### Thank you email +> `%{ISSUE_DESCRIPTION}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223751) in GitLab 16.0. + When a user submits an issue through Service Desk, GitLab sends a **thank you email**. -You must name the template file `thank_you.md`. + +To create a custom email template, in the `.gitlab/service_desk_templates/` +directory in your repository, create a file named `thank_you.md`. You can use these placeholders to be automatically replaced in each email: - `%{ISSUE_ID}`: issue IID - `%{ISSUE_PATH}`: project path appended with the issue IID +- `%{ISSUE_DESCRIPTION}`: issue description based on the original email - `%{UNSUBSCRIBE_URL}`: unsubscribe URL - `%{SYSTEM_HEADER}`: [system header message](../admin_area/appearance.md#system-header-and-footer-messages) - `%{SYSTEM_FOOTER}`: [system footer message](../admin_area/appearance.md#system-header-and-footer-messages) @@ -139,20 +123,28 @@ the response email does not contain the issue link. #### New note email +> `%{ISSUE_DESCRIPTION}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223751) in GitLab 16.0. + When a user-submitted issue receives a new comment, GitLab sends a **new note email**. -You must name the template file `new_note.md`. + +To create a custom email template, in the `.gitlab/service_desk_templates/` +directory in your repository, create a file named `new_note.md`. You can use these placeholders to be automatically replaced in each email: - `%{ISSUE_ID}`: issue IID - `%{ISSUE_PATH}`: project path appended with the issue IID +- `%{ISSUE_DESCRIPTION}`: issue description at the time email is generated. + If a user has edited the description, it might contain sensitive information that is not intended + to be delivered to external participants. Use this placeholder only if you never modify + descriptions or your team is aware of the template design. - `%{NOTE_TEXT}`: note text - `%{UNSUBSCRIBE_URL}`: unsubscribe URL - `%{SYSTEM_HEADER}`: [system header message](../admin_area/appearance.md#system-header-and-footer-messages) - `%{SYSTEM_FOOTER}`: [system footer message](../admin_area/appearance.md#system-header-and-footer-messages) - `%{ADDITIONAL_TEXT}`: [custom additional text](../admin_area/settings/email.md#custom-additional-text) -#### New Service Desk issues +### Use a custom template for Service Desk issues You can select one [description template](description_templates.md#create-an-issue-template) **per project** to be appended to every new Service Desk issue's description. @@ -165,57 +157,65 @@ You can set description templates at various levels: The templates are inherited. For example, in a project, you can also access templates set for the instance, or the project's parent groups. +Prerequisite: + +- You must have [created a description template](description_templates.md#create-an-issue-template). + To use a custom description template with Service Desk: 1. On the top bar, select **Main menu > Projects** and find your project. -1. [Create a description template](description_templates.md#create-an-issue-template). -1. On the left sidebar, select **Settings > General > Service Desk**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. 1. From the dropdown list **Template to append to all Service Desk issues**, search or select your template. -### Using a custom email display name +### Support Bot user + +Behind the scenes, Service Desk works by the special Support Bot user creating issues. +This user isn't a [billable user](../../subscriptions/self_managed/index.md#billable-users), +so it does not count toward the license limit count. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7529) in GitLab 12.8. +#### Change the Support Bot's display name -You can customize the email display name. Emails sent from Service Desk have +You can change the display name of the Support Bot user. Emails sent from Service Desk have this name in the `From` header. The default display name is `GitLab Support Bot`. To edit the custom email display name: 1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General > Service Desk**. -1. Enter a new name in **Email display name**. -1. Select **Save Changes**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Below **Email display name**, enter a new name. +1. Select **Save changes**. -### Using a custom email address **(FREE SELF)** +### Use a custom email address **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284656) in GitLab 13.8. -It is possible to customize the email address used by Service Desk. To do this, you must configure -a [custom mailbox](#configuring-a-custom-mailbox). If you want you can also configure a -[custom suffix](#configuring-a-custom-email-address-suffix). +You can use a custom email address with Service Desk. -#### Configuring a custom mailbox +To do this, you must configure +a [custom mailbox](#configure-a-custom-mailbox). You can also configure a +[custom suffix](#configure-a-custom-email-address-suffix). + +#### Configure a custom mailbox NOTE: On GitLab.com a custom mailbox is already configured with `contact-project+%{key}@incoming.gitlab.com` as the email address, you can still configure the -[custom suffix](#configuring-a-custom-email-address-suffix) in project settings. +[custom suffix](#configure-a-custom-email-address-suffix) in project settings. Service Desk uses the [incoming email](../../administration/incoming_email.md) configuration by default. However, by using the `service_desk_email` configuration, you can customize the mailbox used by Service Desk. This allows you to have -a separate email address for Service Desk by also configuring a [custom suffix](#configuring-a-custom-email-address-suffix) +a separate email address for Service Desk by also configuring a [custom suffix](#configure-a-custom-email-address-suffix) in project settings. -The `address` must include the `+%{key}` placeholder in the 'user' -portion of the address, before the `@`. The placeholder is used to identify the project -where the issue should be created. +Prerequisites: -NOTE: -When configuring a custom mailbox, the `service_desk_email` and `incoming_email` -configurations must always use separate mailboxes. It's important, because -emails picked from `service_desk_email` mailbox are processed by a different -worker and it would not recognize `incoming_email` emails. +- The `address` must include the `+%{key}` placeholder in the `user` portion of the address, + before the `@`. The placeholder is used to identify the project where the issue should be created. +- The `service_desk_email` and `incoming_email` configurations must always use separate mailboxes + to make sure Service Desk emails are processed correctly. To configure a custom mailbox for Service Desk with IMAP, add the following snippets to your configuration file in full: @@ -274,7 +274,7 @@ The configuration options are the same as for configuring > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108279) in GitLab 15.9. Instead of having the Service Desk email credentials stored in plaintext in the configuration files, you can optionally -use an encrypted file for the Incoming email credentials. +use an encrypted file for the incoming email credentials. Prerequisites: @@ -403,7 +403,8 @@ read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secre > - Alternative Azure deployments [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5978) in GitLab 14.9. Service Desk can be configured to read Microsoft Exchange Online mailboxes with the Microsoft -Graph API instead of IMAP. Follow the [documentation in the incoming email section for setting up an OAuth 2.0 application for Microsoft Graph](../../administration/incoming_email.md#microsoft-graph). +Graph API instead of IMAP. Set up an OAuth 2.0 application for Microsoft Graph +[the same way as for incoming email](../../administration/incoming_email.md#microsoft-graph). - Example for Omnibus GitLab installations: @@ -422,32 +423,44 @@ Graph API instead of IMAP. Follow the [documentation in the incoming email secti } ``` -For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), configure the `azure_ad_endpoint` and `graph_endpoint` settings. +For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), +configure the `azure_ad_endpoint` and `graph_endpoint` settings. - Example for Microsoft Cloud for US Government: ```ruby - gitlab_rails['service_desk_email_inbox_options'] = { - 'azure_ad_endpoint': 'https://login.microsoftonline.us', - 'graph_endpoint': 'https://graph.microsoft.us', - 'tenant_id': '<YOUR-TENANT-ID>', - 'client_id': '<YOUR-CLIENT-ID>', - 'client_secret': '<YOUR-CLIENT-SECRET>', - 'poll_interval': 60 # Optional - } +gitlab_rails['service_desk_email_inbox_options'] = { + 'azure_ad_endpoint': 'https://login.microsoftonline.us', + 'graph_endpoint': 'https://graph.microsoft.us', + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional } ``` -The Microsoft Graph API is not yet supported in source installations. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/326169) for more details. +The Microsoft Graph API is not yet supported in source installations. +For more information, see [issue 326169](https://gitlab.com/gitlab-org/gitlab/-/issues/326169). -#### Configuring a custom email address suffix +#### Configure a custom email address suffix -You can set a custom suffix in your project's Service Desk settings after you have configured a [custom mailbox](#configuring-a-custom-mailbox). -It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). +You can set a custom suffix in your project's Service Desk settings. + +A suffix can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). When configured, the custom suffix creates a new Service Desk email address, consisting of the `service_desk_email_address` setting and a key of the format: `<project_full_path>-<custom_suffix>` +Prerequisites: + +- You must have configured a [custom mailbox](#configure-a-custom-mailbox). + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Service Desk**. +1. Below **Email address suffix**, enter the suffix to use. +1. Select **Save changes**. + For example, suppose the `mygroup/myproject` project Service Desk settings has the following configured: - Email address suffix is set to `support`. @@ -456,13 +469,23 @@ For example, suppose the `mygroup/myproject` project Service Desk settings has t The Service Desk email address for this project is: `contact+mygroup-myproject-support@example.com`. The [incoming email](../../administration/incoming_email.md) address still works. -If you don't configure the custom suffix, the default project identification is used for identifying the project. You can see that email address in the project settings. +If you don't configure a custom suffix, the default project identification is used for identifying +the project. -## Using Service Desk +## Use Service Desk You can use Service Desk to [create an issue](#as-an-end-user-issue-creator) or [respond to one](#as-a-responder-to-the-issue). In these issues, you can also see our friendly neighborhood [Support Bot](#support-bot-user). +### View Service Desk email address + +To check what the Service Desk email address is for your project: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues > Service Desk**. + +The email address is available at the top of the issue list. + ### As an end user (issue creator) > Support for additional email headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346600) in GitLab 14.6. In earlier versions, the Service Desk email address had to be in the "To" field. @@ -502,39 +525,51 @@ You can read and write comments as you usually do in GitLab: - The project's visibility (private, internal, public) does not affect Service Desk. - The path to the project, including its group or namespace, is shown in emails. -#### Receiving files attached to comments as email attachments +#### View Service Desk issues -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. Disabled by default. +Prerequisites: -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. -On GitLab.com, this feature is not available. +- You must have at least the Reporter role for the project. -If a comment contains any attachments and their total size is less than or equal to 10 MB, these -attachments are sent as part of the email. In other cases, the email contains links to the attachments. +To view Service Desk issues: -#### Special HTML formatting in HTML emails +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues > Service Desk**. -<!-- When the feature flag is removed, delete this topic and add as a line in version history under one of the topics above this one.--> +### Email contents and formatting -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/372301) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. Disabled by default. +#### Special HTML formatting in HTML emails -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109811) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `service_desk_html_to_text_email_handler`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116809) in GitLab 15.11. Feature flag `service_desk_html_to_text_email_handler` removed. -When this feature is enabled, HTML emails correctly show additional HTML formatting, such as: +HTML emails show HTML formatting, such as: - Tables - Blockquotes - Images - Collapsible sections -#### Privacy considerations +#### Files attached to comments + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11733) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386860) in GitLab 15.10. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. +On GitLab.com, this feature is available. + +If a comment contains any attachments and their total size is less than or equal to 10 MB, these +attachments are sent as part of the email. In other cases, the email contains links to the attachments. + +In GitLab 15.9 and earlier, uploads to a comment are sent as links in the email. + +## Privacy considerations > [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108901) the minimum required role to view the creator's and participant's email in GitLab 15.9. -Service Desk issues are confidential, but the project owner can +Service Desk issues are [confidential](issues/confidential_issues.md), so they are +only visible to project members. The project owner can [make an issue public](issues/confidential_issues.md#modify-issue-confidentiality). When a Service Desk issue becomes public, the issue creator's and participants' email addresses are visible to signed-in users with at least the Reporter role for the project. @@ -542,17 +577,18 @@ visible to signed-in users with at least the Reporter role for the project. In GitLab 15.8 and earlier, when a Service Desk issue becomes public, the issue creator's email address is disclosed to everyone who can view the project. -### Support Bot user +Anyone in your project can use the Service Desk email address to create an issue in this project, **regardless +of their role** in the project. -Behind the scenes, Service Desk works by the special Support Bot user creating issues. This user -does not count toward the license limit count. +The unique internal email address is visible to project members at least +the Reporter role in your GitLab instance. +An external user (issue creator) cannot see the internal email address +displayed in the information note. ### Moving a Service Desk issue > [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/372246) in GitLab 15.7: customers continue receiving notifications when a Service Desk issue is moved. -Service Desk issues can be moved like any other issue in GitLab. - You can move a Service Desk issue the same way you [move a regular issue](issues/managing_issues.md#move-an-issue) in GitLab. @@ -567,8 +603,3 @@ in both issues. They continue to receive any notifications in the old issue and Your emails might be ignored because they contain one of the [email headers that GitLab ignores](../../administration/incoming_email.md#rejected-headers). - -### Responses to a Service Desk issue do not generate emails - -Your issue might have been moved to a different project. -Moved Service Desk issues do not retain email participants. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 3715eef08e0..958246908bc 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -1,46 +1,70 @@ --- stage: Manage -group: Import +group: Import and Integrate info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Migrating projects using file exports **(FREE)** Existing projects on any self-managed GitLab instance or GitLab.com can be exported to a file and -then imported into a new GitLab instance. You can also: +then imported into another GitLab instance. You can also copy GitLab projects to another location with more automation by +[migrating groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). -- [Migrate groups](../../group/import/index.md) using the preferred method. -- [Migrate groups using file exports](../../group/settings/import_export.md). +## Preserving user contributions -GitLab maps user contributions correctly when an admin access token is used to perform the import. +Preserving user contribution depends on meeting the following requirements: -Consequently, migrating projects using file exports does not map user contributions correctly when you are importing -projects from a self-managed instance to GitLab.com. +### Migrating from GitLab self-managed to GitLab.com -Instead, all GitLab user associations (such as comment author) are changed to the user importing the project. For more -information, see the prerequisites and important notes in these sections: +When migrating projects by using file exports, an administrator's access token is required for user contributions to map correctly. -- [Export a project and its data](../settings/import_export.md#export-a-project-and-its-data). -- [Import the project](../settings/import_export.md#import-a-project-and-its-data). +Therefore, user contributions never map correctly when importing file exports from a self-managed instance to GitLab.com. Instead, all GitLab user associations (such as +comment author) are changed to the user importing the project. To preserve contribution history, do one of the following: -To preserve contribution history, [migrate using direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- [Migrate by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- Consider paid GitLab [migration services](https://about.gitlab.com/services/migration/). -If you migrate from GitLab.com to self-managed GitLab, an administrator can create users on the self-managed GitLab instance. +### Migrating to GitLab self-managed + +To ensure GitLab maps users and their contributions correctly: + +- The owner of the project's top-level group should export the project so that the information of all members (direct and inherited) with access to the project can be included in the exported file. Project maintainers and owners can initiate the project export. However, only direct members of a project are then exported. +- An administrator must perform the import with an administrator access token. +- Required users must exist on the destination GitLab instance. An administrator can create confirmed users either in bulk in a Rails console or one by one in the UI. +- Users must [set a public email in their profiles](../../profile/index.md#set-your-public-email) on the source GitLab instance that matches their primary email + address on the destination GitLab instance. You can also manually add users' public emails by + [editing project export files](#edit-project-export-files). + +When the email of an existing user matches the email of an imported user, that user is added as a [direct member](../members/index.md) to the imported project. + +If any of the previous conditions are not met, user contributions are not mapped correctly. Instead, all GitLab user associations are changed to the user who performed the import. +That user becomes an author of merge requests created by other users. Supplementary comments mentioning original authors are: + +- Added for comments, merge request approvals, linked tasks, and items. +- Not added for the merge request or issue creator, added or removed labels, and merged-by information. + +## Edit project export files + +You can add or remove data from export files. For example, you can: + +- Manually add users public emails to the `project_members.ndjson` file. +- Trim CI pipelines by removing lines from the `ci_pipelines.ndjson` file. + +To edit a project export file: + +1. Extract the exported `.tar.gz` file. +1. Edit the appropriate file . For example, `tree/project/project_members.ndjson`. +1. Compress the files back to a `.tar.gz` file. + +You can also make sure that all members were exported by checking the `project_members.ndjson` file. ## Compatibility -FLAG: -On self-managed GitLab by default project, file exports are in NDJSON format. To make GitLab produce project file -exports in JSON format, ask an administrator to [disable the feature flags](../../../administration/feature_flags.md) -named `project_export_as_ndjson`. To allow GitLab to import project file exports in JSON format, ask an administrator to -[disable the feature flags](../../../administration/feature_flags.md) named `project_import_ndjson`. On GitLab.com, -project file exports are in NDJSON format only. +> Support for JSON-formatted project file exports [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/389888) in GitLab 15.11. -Project file exports are in NDJSON format. Before version 14.0, GitLab produced project file exports in JSON format. -To support transitions, you can still import JSON-formatted project file exports if you configure the relevant feature -flags. +Project file exports are in NDJSON format. -From GitLab 13.0, GitLab can import project file exports that were exported from a version of GitLab up to two +You can import project file exports that were exported from a version of GitLab up to two [minor](../../../policy/maintenance.md#versioning) versions behind, which is similar to our process for [security releases](../../../policy/maintenance.md#security-releases). @@ -85,7 +109,6 @@ Prerequisites: - Review the list of [items that are exported](#items-that-are-exported). Not all items are exported. - You must have at least the Maintainer role for the project. -- Users must [set a public email](../../profile/index.md#set-your-public-email) in the source GitLab instance that matches one of their verified emails in the target GitLab instance for the user mapping to work correctly. To export a project and its data, follow these steps: @@ -103,76 +126,86 @@ moved to your configured `uploads_directory`. Every 24 hours, a worker deletes t ### Items that are exported -The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) -file for projects lists many of the items exported and imported when migrating projects using file exports. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, -[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/project/import_export.yml). +Exported project items depend on the version of GitLab you use. To determine if a +specific project item is exported: -Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and -[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. +1. Check the [`exporters` array](https://gitlab.com/gitlab-org/gitlab/blob/0a60d6dcfa7b809cf4fe0c2e239f406014d92e34/app/services/projects/import_export/export_service.rb#L25-28). +1. Check the [`project/import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) file for projects for your GitLab version (for example, `<https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/gitlab/import_export/project/import_export.yml>` for GitLab 15.9). -Items that are exported include: +For a quick overview, items that are exported include: - Project and wiki repositories - Project uploads - Project configuration, excluding integrations - Issues - Issue comments - - Issue iteration ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in 15.4) - - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource iteration events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue iterations ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in GitLab 15.4) + - Issue resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource milestone events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource iteration events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Merge requests - Merge request diffs - Merge request comments - - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Merge request multiple assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Merge request multiple assignees ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request reviewers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request approvers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) +- Commit comments ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) in GitLab 15.10) - Labels - Milestones - Snippets - Time tracking and other project entities -- Design Management files and data +- Design management files and data - LFS objects - Issue boards - Pipelines history -- Push Rules -- Awards -- Group members are exported as project members, as long as the user has the Maintainer role in the - exported project's group, or is an administrator +- Push rules +- Emoji reactions +- Group members as long as the user has the Maintainer role in the + exported project's group or is an administrator + +### Items that are not exported Items that are **not** exported include: - [Child pipeline history](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) +- Pipeline triggers - Build traces and artifacts - Package and container registry images - CI/CD variables -- Pipeline triggers - Webhooks - Any encrypted tokens - [Number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221087) - Repository size limits - Deploy keys allowed to push to protected branches -- Secure Files -- [Activity logs for Git-related events](https://gitlab.com/gitlab-org/gitlab/-/issues/214700). For example, pushing and creating tags +- Secure files +- [Activity logs for Git-related events](https://gitlab.com/gitlab-org/gitlab/-/issues/214700) (for example, pushing and creating tags) +- Security policies associated with your project + +Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and +[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. ## Import a project and its data -> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. +> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. Administrators of self-managed instances can [set maximum import file size](#set-maximum-import-file-size). On GitLab.com, the value is [set to 5 GB](../../gitlab_com/index.md#account-and-limit-settings). + +You can import a project and its data. WARNING: Only import projects from sources you trust. If you import a project from an untrusted source, it may be possible for an attacker to steal your sensitive data. -Prerequisites: +### Prerequisites + +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. - You must have [exported the project and its data](#export-a-project-and-its-data). - Compare GitLab versions and ensure you are importing to a GitLab version that is the same or later than the GitLab version you exported to. - Review [compatibility](#compatibility) for any issues. -- At least the Maintainer role on the destination group to migrate to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +- At least the Maintainer role on the destination group to migrate to. + +### Import a project To import a project: @@ -200,14 +233,9 @@ Deploy keys aren't imported. To use deploy keys, you must enable them in your im ### Import large projects **(FREE SELF)** -If you have a larger project, consider using a Rake task as described in the [developer documentation](../../../development/import_project.md#importing-via-a-rake-task). +If you have a larger project, consider [using a Rake task](../../../administration/raketasks/project_import_export.md#import-large-projects). -## Automate group and project import **(PREMIUM)** - -For information on automating user, group, and project import API calls, see -[Automate group and project import](../import/index.md#automate-group-and-project-import). - -## Maximum import file size +## Set maximum import file size **(FREE SELF)** Administrators can set the maximum import file size one of two ways: @@ -216,34 +244,15 @@ Administrators can set the maximum import file size one of two ways: The default is `0` (unlimited). -For the GitLab.com setting, see the [Account and limit settings](../../gitlab_com/index.md#account-and-limit-settings) -section of the GitLab.com settings page. - -## Map users for import - -Imported users can be mapped by their public email addresses on self-managed instances, if an administrator (not an owner) does the import. - -- The project must be exported by a project or group member with the Owner role. -- Public email addresses are not set by default. Users must [set it in their profiles](../../profile/index.md#set-your-public-email) - for mapping to work correctly. -- For contributions to be mapped correctly, users must be an existing member of the namespace, - or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original - author and the merge requests, notes, or issues that are owned by the importer. -- Imported users are set as [direct members](../members/index.md) - in the imported project. - -For project migration imports performed over GitLab.com groups, preserving author information is -possible through a [professional services engagement](https://about.gitlab.com/services/migration/). - ## Rate limits To help avoid abuse, by default, users are rate limited to: -| Request Type | Limit | -| ---------------- | ----- | -| Export | 6 projects per minute | -| Download export | 1 download per group per minute | -| Import | 6 projects per minute | +| Request type | Limit | +|:----------------|:--------------------------------| +| Export | 6 projects per minute | +| Download export | 1 download per group per minute | +| Import | 6 projects per minute | ## Related topics @@ -251,3 +260,5 @@ To help avoid abuse, by default, users are rate limited to: - [Project import and export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Migrating GitLab groups](../../group/import/index.md) - [Group import and export API](../../../api/group_import_export.md) +- [Migrate groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- [Migrate groups by using file exports](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). diff --git a/doc/user/project/settings/import_export_troubleshooting.md b/doc/user/project/settings/import_export_troubleshooting.md index 00edeef5cfa..c3abfa015a6 100644 --- a/doc/user/project/settings/import_export_troubleshooting.md +++ b/doc/user/project/settings/import_export_troubleshooting.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Import +group: Import and Integrate info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- @@ -10,10 +10,10 @@ If you have problems with [migrating projects using file exports](import_export. ## Troubleshooting commands -Finds information about the status of the import and further logs using the JID: +Finds information about the status of the import and further logs using the JID, +using the [Rails console](../../../administration/operations/rails_console.md): ```ruby -# Rails console Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error) > {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil} ``` @@ -35,6 +35,27 @@ Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and - Ensure shared runners are enabled in both the source and destination projects. - Disable shared runners on the parent group when you import the project. +## Users missing from imported project + +If users aren't imported with imported projects, see the [preserving user contributions](import_export.md#preserving-user-contributions) requirements. + +A common reason for missing users is that the [public email setting](../../profile/index.md#set-your-public-email) isn't configured for users. +To resolve this issue, ask users to configure this setting using the GitLab UI. + +If there are too many users for manual configuration to be feasible, +you can set all user profiles to use a public email address using the +[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + +```ruby +User.where("public_email IS NULL OR public_email = '' ").find_each do |u| + next if u.bot? + + puts "Setting #{u.username}'s currently empty public email to #{u.email}…" + u.public_email = u.email + u.save! +end +``` + ## Import workarounds for large repositories [Maximum import size limitations](import_export.md#import-a-project-and-its-data) @@ -48,41 +69,41 @@ reduce the repository size for another import attempt: 1. Create a temporary working directory from the export: - ```shell - EXPORT=<filename-without-extension> + ```shell + EXPORT=<filename-without-extension> - mkdir "$EXPORT" - tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ - cd "$EXPORT"/ - git clone project.bundle + mkdir "$EXPORT" + tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ + cd "$EXPORT"/ + git clone project.bundle - # Prevent interference with recreating an importable file later - mv project.bundle ../"$EXPORT"-original.bundle - mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz + # Prevent interference with recreating an importable file later + mv project.bundle ../"$EXPORT"-original.bundle + mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz - git switch --create smaller-tmp-main - ``` + git switch --create smaller-tmp-main + ``` 1. To reduce the repository size, work on this `smaller-tmp-main` branch: [identify and remove large files](../repository/reducing_the_repo_size_using_git.md) or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase) to reduce the number of commits. - ```shell - # Reduce the .git/objects/pack/ file size - cd project - git reflog expire --expire=now --all - git gc --prune=now --aggressive - - # Prepare recreating an importable file - git bundle create ../project.bundle <default-branch-name> - cd .. - mv project/ ../"$EXPORT"-project - cd .. - - # Recreate an importable file - tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . - ``` + ```shell + # Reduce the .git/objects/pack/ file size + cd project + git reflog expire --expire=now --all + git gc --prune=now --aggressive + + # Prepare recreating an importable file + git bundle create ../project.bundle <default-branch-name> + cd .. + mv project/ ../"$EXPORT"-project + cd .. + + # Recreate an importable file + tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . + ``` 1. Import this new, smaller file into GitLab. 1. In a full clone of the original repository, @@ -153,12 +174,12 @@ Rather than attempting to push all changes at once, this workaround: You usually export a project through [the web interface](import_export.md#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these methods can sometimes fail without giving enough information to troubleshoot. In these cases, -[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) and loop through +[open a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) and loop through [all the defined exporters](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/projects/import_export/export_service.rb). Execute each line individually, rather than pasting the entire block at once, so you can see any errors each command returns. -```shell +```ruby # User needs to have permission to export u = User.find_by_username('someuser') p = Project.find_by_full_path('some/project') @@ -265,12 +286,12 @@ Marked stuck import jobs as failed. JIDs: xyz | Problem | Possible solutions | | -------- | -------- | | [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) | -| | Batch export -| | Optimize SQL -| | Move away from `ActiveRecord` callbacks (difficult) -| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory | +| | Batch export | +| | Optimize SQL | +| | Move away from `ActiveRecord` callbacks (difficult) | +| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857)) | DB Commit sweet spot that uses less memory | | | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help | -| | Batch reading/writing to disk and any SQL +| | Batch reading/writing to disk and any SQL | ### Temporary solutions diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index ed4eea56514..f89c2a1eaaa 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: 'To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments' type: reference, index, howto --- @@ -75,28 +75,28 @@ To configure visibility, features, and permissions for a project: Use the toggles to enable or disable features in the project. -| Option | More access limit options | Description | -| :------------------------------- | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Issues** | ✓ | Activates the GitLab issues tracker. | -| **Repository** | ✓ | Enables [repository](../repository/index.md) functionality | -| **Merge requests** | ✓ | Enables [merge request](../merge_requests/index.md) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). | -| **Forks** | ✓ | Enables [forking](../repository/forking_workflow.md) functionality. | -| **Git Large File Storage (LFS)** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs). | -| **Packages** | | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality. | -| **CI/CD** | ✓ | Enables [CI/CD](../../../ci/index.md) functionality. | -| **Container Registry** | | Activates a [registry](../../packages/container_registry/index.md) for your Docker images. | -| **Analytics** | ✓ | Enables [analytics](../../analytics/index.md). | -| **Requirements** | ✓ | Control access to [Requirements Management](../requirements/index.md). | -| **Security & Compliance** | ✓ | Control access to [security features](../../application_security/index.md). | -| **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/index.md). | -| **Snippets** | ✓ | Enables [sharing of code and text](../../snippets.md). | -| **Pages** | ✓ | Allows you to [publish static websites](../pages/index.md). | -| **Metrics Dashboard** | ✓ | Control access to [metrics dashboard](../integrations/prometheus.md). | -| **Releases** | ✓ | Control access to [Releases](../releases/index.md). | -| **Environments** | ✓ | Control access to [Environments and Deployments](../../../ci/environments/index.md). | -| **Feature flags** | ✓ | Control access to [Feature Flags](../../../operations/feature_flags.md). | -| **Monitor** | ✓ | Control access to [Monitor](../../../operations/index.md) features. | -| **Infrastructure** | ✓ | Control access to [Infrastructure](../../infrastructure/index.md) features. | +| Option | More access limit options | Description +| :------------------------------- | :------------------------ | :---------- | +| **Issues** | **{check-circle}** Yes | Activates the GitLab issues tracker. +| **Repository** | **{check-circle}** Yes | Enables [repository](../repository/index.md) functionality. +| **Merge requests** | **{check-circle}** Yes | Enables [merge request](../merge_requests/index.md) functionality; also see [Merge request settings](#configure-merge-request-settings-for-a-project). +| **Forks** | **{check-circle}** Yes | Enables [forking](../repository/forking_workflow.md) functionality. +| **Git Large File Storage (LFS)** | **{dotted-circle}** No | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs). +| **Packages** | **{dotted-circle}** No | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality. +| **CI/CD** | **{check-circle}** Yes | Enables [CI/CD](../../../ci/index.md) functionality. +| **Container Registry** | **{dotted-circle}** No | Activates a [registry](../../packages/container_registry/index.md) for your Docker images. +| **Analytics** | **{check-circle}** Yes | Enables [analytics](../../analytics/index.md). +| **Requirements** | **{check-circle}** Yes | Control access to [Requirements Management](../requirements/index.md). +| **Security and Compliance** | **{check-circle}** Yes | Control access to [security features](../../application_security/index.md). +| **Wiki** | **{check-circle}** Yes | Enables a separate system for [documentation](../wiki/index.md). +| **Snippets** | **{check-circle}** Yes | Enables [sharing of code and text](../../snippets.md). +| **Pages** | **{check-circle}** Yes | Allows you to [publish static websites](../pages/index.md). +| **Metrics Dashboard** | **{check-circle}** Yes | Control access to [metrics dashboard](../integrations/prometheus.md). +| **Releases** | **{check-circle}** Yes | Control access to [Releases](../releases/index.md). +| **Environments** | **{check-circle}** Yes | Control access to [Environments and Deployments](../../../ci/environments/index.md). +| **Feature flags** | **{check-circle}** Yes | Control access to [Feature flags](../../../operations/feature_flags.md). +| **Monitor** | **{check-circle}** Yes | Control access to [Monitor](../../../operations/index.md) features. +| **Infrastructure** | **{check-circle}** Yes | Control access to [Infrastructure](../../infrastructure/index.md) features. When you disable a feature, the following additional features are also disabled: @@ -162,6 +162,7 @@ Configure your project's merge request settings: - Configure [suggested changes commit messages](../merge_requests/reviews/suggestions.md#configure-the-commit-message-for-applied-suggestions). - Configure [merge and squash commit message templates](../merge_requests/commit_templates.md). - Configure [the default target project](../merge_requests/creating_merge_requests.md#set-the-default-target-project) for merge requests coming from forks. +- Enable [Suggested Reviewers](../merge_requests/reviews/index.md#suggested-reviewers). ## Service Desk @@ -174,7 +175,7 @@ Learn how to [export a project](import_export.md#import-a-project-and-its-data) ## Advanced project settings Use the advanced settings to archive, rename, transfer, -remove a fork relationship, or delete a project. +[remove a fork relationship](../repository/forking_workflow.md#unlink-a-fork), or delete a project. ### Archive a project @@ -347,24 +348,6 @@ To restore a project marked for deletion: 1. Navigate to your project, and select **Settings > General > Advanced**. 1. In the Restore project section, select **Restore project**. -## Remove a fork relationship - -Prerequisites: - -- You must be a project owner to remove a fork relationship. - -WARNING: -If you remove a fork relationship, you can't send merge requests to the source. If anyone has forked your project, their fork also loses the relationship. -To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). - -To remove a fork relationship: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Advanced**. -1. In the **Remove fork relationship** section, select **Remove fork relationship**. -1. To confirm, enter the project path and select **Confirm**. - ## Monitor settings ### Alerts @@ -375,7 +358,7 @@ Configure [alert integrations](../../../operations/incident_management/integrati #### Alert integration -Automatically [create](../../../operations/metrics/alerts.md#trigger-actions-from-alerts), [notify on](../../../operations/incident_management/paging.md#email-notifications-for-alerts), and [resolve](../../../operations/incident_management/manage_incidents.md#automatically-close-incidents-via-recovery-alerts) incidents based on GitLab alerts. +Automatically [create](../../../operations/incident_management/alerts.md#trigger-actions-from-alerts), [notify on](../../../operations/incident_management/paging.md#email-notifications-for-alerts), and [resolve](../../../operations/incident_management/manage_incidents.md#automatically-close-incidents-via-recovery-alerts) incidents based on GitLab alerts. #### PagerDuty integration @@ -398,16 +381,6 @@ to enable the syncing of public Issues to a [deployed status page](../../../oper When working with project settings, you might encounter the following issues, or require alternate methods to complete specific tasks. -### Remove a fork relationship through console - -If removing the fork through the UI or API is not working, you can attempt the fork relationship removal in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session). - -```ruby -p = Project.find_by_full_path('<project_path>') -u = User.find_by_username('<username>') -Projects::UnlinkForkService.new(p, u).execute -``` - ### Transfer a project through console If transferring a project through the UI or API is not working, you can attempt the transfer in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session). diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index f9218a228ca..178500093e2 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -28,16 +28,12 @@ and [personal access tokens](../../profile/personal_access_tokens.md). In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. WARNING: -The ability to create project access tokens without expiry was -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and is planned for removal in GitLab -16.0. When this ability is removed, existing project access tokens without an expiry are planned to have an expiry added. -The automatic adding of an expiry occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry -occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. +The ability to create project access tokens without expiry was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. In GitLab 16.0 and later, existing project access tokens without an expiry date are automatically given an expiry date of 365 days later than the current date. The automatic adding of an expiry date occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry date occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. You can use project access tokens: -- On GitLab SaaS if you have the Premium license tier or higher. Project access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). -- On self-managed instances of GitLab, with any license tier. If you have the Free tier: +- On GitLab SaaS: If you have the Premium or Ultimate license tier. Project access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). +- On self-managed instances of GitLab: With any license tier. If you have the Free tier: - Review your security and compliance policies around [user self-enrollment](../../admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups). - Consider [disabling project access tokens](#enable-or-disable-project-access-token-creation) to @@ -48,32 +44,33 @@ You cannot use project access tokens to create other group, project, or personal Project access tokens inherit the [default prefix setting](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) configured for personal access tokens. -NOTE: -Project access tokens are not FIPS compliant and creation and use are disabled when [FIPS mode](../../../development/fips_compliance.md) is enabled. - ## Create a project access token > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89114) in GitLab 15.1, Owners can select Owner role for project access tokens. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348660) in GitLab 15.3, default expiration of 30 days and default role of Guest is populated in the UI. +> - Ability to create non-expiring project access tokens [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. + +WARNING: +Project access tokens are treated as [internal users](../../../development/internal_users.md). +If an internal user creates a project access token, that token is able to access +all projects that have visibility level set to [Internal](../../public_access.md). To create a project access token: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the project. -1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. - +1. Enter an expiry date for the token. + - The token expires on that date at midnight UTC. + - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. + - By default, this date can be a maximum of 365 days later than the current date. + - An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-project-access-token). 1. Select **Create project access token**. A project access token is displayed. Save the project access token somewhere safe. After you leave or refresh the page, you can't view it again. -WARNING: -Project access tokens are treated as [internal users](../../../development/internal_users.md). -If an internal user creates a project access token, that token is able to access -all projects that have visibility level set to [Internal](../../public_access.md). - ## Revoke a project access token To revoke a project access token: @@ -121,12 +118,8 @@ The bot users for projects have [permissions](../../permissions.md#project-membe selected role and [scope](#scopes-for-a-project-access-token) of the project access token. - The name is set to the name of the token. -- The username is set to `project_{project_id}_bot` for the first access token. For example, `project_123_bot`. -- The email is set to `project{project_id}_bot@noreply.{Gitlab.config.gitlab.host}`. For example, `project123_bot@noreply.example.com`. -- For additional access tokens in the same project, the username is set to `project_{project_id}_bot{bot_count}`. For - example, `project_123_bot1`. -- For additional access tokens in the same project, the email is set to `project{project_id}_bot{bot_count}@noreply.{Gitlab.config.gitlab.host}`. - For example, `project123_bot1@noreply.example.com`. +- The username is set to `project_{project_id}_bot_{random_string}`. For example, `project_123_bot_4ffca233d8298ea1`. +- The email is set to `project_{project_id}_bot_{random_string}@noreply.{Gitlab.config.gitlab.host}`. For example, `project_123_bot_4ffca233d8298ea1@noreply.example.com`. API calls made with a project access token are associated with the corresponding bot user. @@ -143,3 +136,7 @@ When the project access token is [revoked](#revoke-a-project-access-token): - All records are moved to a system-wide user with the username [Ghost User](../../profile/account/delete_account.md#associated-records). See also [Bot users for groups](../../group/settings/group_access_tokens.md#bot-users-for-groups). + +## Token availability + +Project access tokens are only available in paid subscriptions, and not available in trial subscriptions. For more information, see the ["What is included" section of the GitLab Trial FAQ](https://about.gitlab.com/free-trial/#what-is-included-in-my-free-trial-what-is-excluded). diff --git a/doc/user/project/system_notes.md b/doc/user/project/system_notes.md new file mode 100644 index 00000000000..1eee4364d95 --- /dev/null +++ b/doc/user/project/system_notes.md @@ -0,0 +1,56 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# System notes **(FREE)** + +System notes are short descriptions that help you understand the history of events +that occur during the life cycle of a GitLab object, such as: + +- [Alerts](../../operations/incident_management/alerts.md). +- [Designs](issues/design_management.md). +- [Issues](issues/index.md). +- [Merge requests](merge_requests/index.md). +- [Objectives and key results](../okrs.md) (OKRs). +- [Tasks](../tasks.md). + +GitLab logs information about events triggered by Git or the GitLab application +in system notes. System notes use the format `<Author> <action> <time ago>`. + +## Show or filter system notes + +By default, system notes do not display. When displayed, they are shown oldest first. +If you change the filter or sort options, your selection is remembered across sections. +The filtering options are: + +- **Show all activity** displays both comments and history. +- **Show comments only** hides system notes. +- **Show history only** hides user comments. + +### On an epic + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Epics** (**{epic}**). +1. Identify your desired epic, and select its title. +1. Go to the **Activity** section. +1. For **Sort or filter**, select **Show all activity**. + +### On an issue + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Issues** and find your issue. +1. Go to **Activity**. +1. For **Sort or filter**, select **Show all activity**. + +### On a merge request + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. Go to **Activity**. +1. For **Sort or filter**, select **Show all activity**. + +## Related topics + +- [Notes API](../../api/notes.md) diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index d5f0b7e6180..bd935db6f02 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -1,6 +1,5 @@ --- type: reference -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/time_tracking.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments diff --git a/doc/user/project/web_ide/img/admin_live_preview_v13_0.png b/doc/user/project/web_ide/img/admin_live_preview_v13_0.png Binary files differdeleted file mode 100644 index 90129d240bc..00000000000 --- a/doc/user/project/web_ide/img/admin_live_preview_v13_0.png +++ /dev/null diff --git a/doc/user/project/web_ide/img/commit_changes_v13_11.png b/doc/user/project/web_ide/img/commit_changes_v13_11.png Binary files differdeleted file mode 100644 index 6cd270a6112..00000000000 --- a/doc/user/project/web_ide/img/commit_changes_v13_11.png +++ /dev/null diff --git a/doc/user/project/web_ide/img/dark_theme_v13_0.png b/doc/user/project/web_ide/img/dark_theme_v13_0.png Binary files differdeleted file mode 100644 index 020578a9444..00000000000 --- a/doc/user/project/web_ide/img/dark_theme_v13_0.png +++ /dev/null diff --git a/doc/user/project/web_ide_beta/img/fuzzy_finder_v15_7.png b/doc/user/project/web_ide/img/fuzzy_finder_v15_7.png Binary files differindex 66ebae15e98..66ebae15e98 100644 --- a/doc/user/project/web_ide_beta/img/fuzzy_finder_v15_7.png +++ b/doc/user/project/web_ide/img/fuzzy_finder_v15_7.png diff --git a/doc/user/project/web_ide/img/live_preview_v13_0.png b/doc/user/project/web_ide/img/live_preview_v13_0.png Binary files differdeleted file mode 100644 index f701e137a6b..00000000000 --- a/doc/user/project/web_ide/img/live_preview_v13_0.png +++ /dev/null diff --git a/doc/user/project/web_ide/img/solarized_dark_theme_v13_1.png b/doc/user/project/web_ide/img/solarized_dark_theme_v13_1.png Binary files differdeleted file mode 100644 index 8eca352a4d0..00000000000 --- a/doc/user/project/web_ide/img/solarized_dark_theme_v13_1.png +++ /dev/null diff --git a/doc/user/project/web_ide/img/terminal_status.png b/doc/user/project/web_ide/img/terminal_status.png Binary files differdeleted file mode 100644 index 91c341a9854..00000000000 --- a/doc/user/project/web_ide/img/terminal_status.png +++ /dev/null diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 5e9f648daba..bb1609a74e5 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -1,472 +1,205 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Web IDE **(FREE)** -The Web IDE is an advanced editor with commit staging. -You can use the Web IDE to make changes to multiple files directly from the -GitLab UI. For a more basic implementation, see [Web Editor](../repository/web_editor.md). - -NOTE: -The Web IDE is being updated to use VS Code. For details, -see [Web IDE Beta](../web_ide_beta/index.md). - -## Open the Web IDE - -Use the <kbd>.</kbd> [keyboard shortcut](../../shortcuts.md) to open the Web IDE. -You can also open the Web IDE when viewing a file, from the repository file list, -and from merge requests: - -### When viewing a file or the repository file list - - 1. In the upper right corner of the page, select **Open in Web IDE** if it is visible. - 1. If **Open in Web IDE** is not visible: - 1. Select the (**{chevron-lg-down}**) next to **Edit** or **Gitpod**, depending on your configuration. - 1. Select **Open in Web IDE** from the list to display it as the editing option. - 1. Select **Open in Web IDE** to open the editor. - -### When viewing a merge request - - 1. Go to your merge request. - 1. In the upper right corner, select **Code > Open in Web IDE**. - -## File finder - -The file finder allows you to quickly open files in the current branch by -searching for fragments of the file path. The file finder is launched using the keyboard shortcut -<kbd>Command</kbd>+<kbd>p</kbd>, <kbd>Control</kbd>+<kbd>p</kbd>, or <kbd>t</kbd> -(when editor is not in focus). Type the filename or file path fragments to -start seeing results. - -## Command palette +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/371084) in GitLab 15.7. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. -You can see all available commands for manipulating editor content by pressing -the <kbd>F1</kbd> key when the editor is in focus. After that, the editor displays -a complete list of available commands for -manipulating editor content. The editor supports commands for multi-cursor -editing, code block folding, commenting, searching and replacing, navigating -editor warnings and suggestions, and more. +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. -Some commands have a keyboard shortcut assigned to them. The command palette -displays this shortcut next to each command. You can use this shortcut to invoke -the command without having to select it in the command palette. - -For a full list of keyboard shortcuts in the Web IDE, refer to the -[Keyboard shortcuts](../../shortcuts.md#web-ide) list. - -## Syntax highlighting - -As expected from an IDE, syntax highlighting for many languages in -the Web IDE makes your direct editing even easier. - -The Web IDE currently provides: - -- Basic syntax colorization for a variety of programming, scripting and markup - languages such as XML, PHP, C#, C++, Markdown, Java, VB, Batch, Python, Ruby, - and Objective-C. -- IntelliSense and validation support (displaying errors and warnings, providing - smart completions, formatting, and outlining) for some languages. For example: - TypeScript, JavaScript, CSS, LESS, SCSS, JSON, and HTML. - -Because the Web IDE is based on the [Monaco Editor](https://microsoft.github.io/monaco-editor/), -you can find a more complete list of supported languages in the -[Monaco languages](https://github.com/Microsoft/monaco-languages) repository. Under the hood, -Monaco uses the [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for syntax highlighting. +The Web IDE is an advanced editor with commit staging. +You can use the Web IDE to make changes to multiple files directly from the GitLab UI. +For a more basic implementation, see [Web Editor](../repository/web_editor.md). -If you are missing Syntax Highlighting support for any language, we prepared a short guide on how to [add support for a missing language Syntax Highlighting.](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/ide/lib/languages/README.md) +To pair the Web IDE with a remote development environment, see [remote development](../remote_development/index.md). -### Themes +## Use the Web IDE -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in GitLab 13.0. -> - Full Solarized Dark Theme [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219228) in GitLab 13.1. -> - Full [Solarized Light](https://gitlab.com/gitlab-org/gitlab/-/issues/221035) and [Monokai](https://gitlab.com/gitlab-org/gitlab/-/issues/221034) Themes introduced in GitLab 13.6. +To open the Web IDE from the GitLab UI: -All the themes GitLab supports for syntax highlighting are applied to the entire Web IDE screen. -You can pick a theme from your [profile preferences](../../profile/preferences.md). +1. On the top bar, select **Main menu > Projects** and find your project. +1. Use the <kbd>.</kbd> keyboard shortcut. -| Solarized Dark Theme | Dark Theme | -|-------------------------------------------------------------|-----------------------------------------| -| ![Solarized Dark Theme](img/solarized_dark_theme_v13_1.png) | ![Dark Theme](img/dark_theme_v13_0.png) | +You can also open the Web IDE from: -## Link to specific lines +- A file +- The repository file list +- A merge request -The Web IDE and the [Web Editor](../repository/web_editor.md) share the -same core features. To link to specific lines in the Web IDE, see -[Web Editor](../repository/web_editor.md#link-to-specific-lines). +### From a file or the repository file list -## Schema based validation +To open the Web IDE from a file or the repository file list: -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218472) validation based on predefined schemas in GitLab 13.2 [with a flag](../../../administration/feature_flags.md) named `schema_linting`. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104399) in GitLab 15.7. -> - On self-managed GitLab [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107488) in GitLab 15.8. +- In the upper-right corner of the page, select **Open in Web IDE**. -The Web IDE provides validation support for certain JSON and YAML files using schemas -based on the [JSON Schema Store](https://www.schemastore.org/json/). +If **Open in Web IDE** is not visible: -### Predefined schemas +1. Next to **Edit** or **Gitpod**, select the down arrow (**{chevron-lg-down}**). +1. From the dropdown list, select **Open in Web IDE**. +1. Select **Open in Web IDE**. -The Web IDE has validation for certain files built in. This feature is only supported for -the `*.gitlab-ci.yml` files. +### From a merge request -### Custom schemas **(PREMIUM)** +To open the Web IDE from a merge request: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/226982) in GitLab 13.4. +1. Go to your merge request. +1. In the upper-right corner, select **Code > Open in Web IDE**. -The Web IDE also allows you to define custom schemas for certain JSON/YAML files in your project. -You can do so by defining a `schemas` entry in the `.gitlab/.gitlab-webide.yml` file inside the -repository's root. Here is an example configuration: +The Web IDE opens new and modified files in separate tabs and displays changes side by side with the original source. +To optimize loading time, only the top 10 files (by number of lines changed) are opened automatically. -```yaml -schemas: - - uri: https://json.schemastore.org/package - match: - - package.json - - uri: https://somewebsite.com/first/raw/url - match: - - data/release_posts/unreleased/*.{yml,yaml} - - uri: https://somewebsite.com/second/raw/url - match: - - "*.meta.json" -``` +In the file tree, any new or modified file in the merge request is indicated by an icon next to the filename. +To view changes to a file, right-click the filename and select **Compare with merge request base**. -Each schema entry supports two properties: +## Open a file in the Web IDE -- `uri`: Provide an absolute URL for the schema definition file here. - The schema from this URL is loaded when a matching file is open. -- `match`: A list of matching paths or glob expressions. If a schema matches a - particular path pattern, it is applied to that file. Enclose the pattern - in quotes if it begins with an asterisk (`*`), it's be applied to that file. - If a pattern begins with an asterisk (`*`), enclose it in quotation marks. - Otherwise, the configuration file is not valid YAML. +To open any file by its name: -## Configure the Web IDE +1. Press <kbd>Command</kbd>+<kbd>P</kbd>. +1. Enter the name of your file. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23352) in GitLab 13.1. +![fuzzy_finder_v15_7](img/fuzzy_finder_v15_7.png) -The Web IDE supports configuration of certain editor settings by using -[`.editorconfig` files](https://editorconfig.org/). When opening a file, the -Web IDE looks for a file named `.editorconfig` in the current directory -and all parent directories. If a configuration file is found and has settings -that match the file's path, these settings are enforced on the opened file. +## Search across files -The Web IDE currently supports the following `.editorconfig` settings: +You can use the Web IDE to search all files in the opened folder. -- `indent_style` -- `indent_size` -- `end_of_line` -- `trim_trailing_whitespace` -- `tab_width` -- `insert_final_newline` +To search across files: -## Commit changes +1. Press <kbd>Shift</kbd>+<kbd>Command</kbd>+<kbd>F</kbd>. +1. Enter your search term. -> - [Starting](https://gitlab.com/gitlab-org/gitlab/-/issues/33441) with GitLab 12.7, files are automatically staged. -> - In GitLab 12.9, support for staging files was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/196609) to prevent loss of unstaged data. All of your current changes must be committed or discarded. +In the Web IDE, only partial results from opened files are displayed. -After making your changes, select **Commit** on the bottom-left to -review the list of changed files. +## View a list of changed files -After you have finalized your changes, you can add a commit message, commit the -changes and directly create a merge request. In case you don't have write -access to the selected branch, you see a warning, but can still create -a new branch and start a merge request. +To view a list of files you changed in the Web IDE: -To discard a change in a particular file, select **Discard changes** on that -file in the changes tab. To discard all the changes, select the trash icon in the -upper-right corner of the changes sidebar. +- On the activity bar on the left, select **Source Control**, + or press <kbd>Control</kbd>+<kbd>Shift</kbd>+<kbd>G</kbd>. -![Commit changes](img/commit_changes_v13_11.png) +Your `CHANGES`, `STAGED CHANGES`, and `MERGE CHANGES` are displayed. +For more information, see the [VS Code documentation](https://code.visualstudio.com/docs/sourcecontrol/overview#_commit). -## Reviewing changes +## Upload a new file -Before you commit your changes, you can compare them with the previous commit -by switching to the review mode or selecting the file from the list of changes. +To upload a new file and add it to the Git repository: -An additional review mode is available when you open a merge request, which -shows you a preview of the merge request diff if you commit your changes. +1. In the **Explorer** file tree, navigate to the directory where you want to upload the file. +1. Optional. If the directory does not exist yet, select the directory path where you want to have a new directory and either: + - Right-click on the directory path, and select **New Folder...**. You can create a nested directory path with the `/` separator, for example `parentdir/subdir1/subdir2`. + - In the **Explorer** panel, in the upper-right corner, select the new folder (**{folder-new}**) icon. +1. Enter the name of the new directory, and press <kbd>Enter/Return</kbd> to create it. +1. Right-click on the directory path and select `Upload...`. +1. Select the file you want to upload, then select `Open`. You can select and add multiple files at once. -## View CI job logs +The file is uploaded and automatically added as a new file to the Git repository. -You can use the Web IDE to quickly fix failing tests by opening -the branch or merge request in the Web IDE and opening the logs of the failed -job. You can access the status of all jobs for the most recent pipeline and job -traces for the current commit by selecting the **Pipelines** button in the top -right. +## Switch branches -The pipeline status is also shown at all times in the status bar in the bottom -left. +The Web IDE uses the currently selected branch by default. +To switch branches in the Web IDE: -## Switching merge requests +1. On the status bar, in the lower-left corner, select the current branch name. +1. In the search box, start typing the branch name. +1. From the dropdown list, select the branch. -To switch between your authored and assigned merge requests, select the -dropdown list in the top of the sidebar to open a list of merge requests. You must commit or discard all your changes before switching to a different merge -request. +## Create a branch -## Switching branches +To create a branch from the current branch in the Web IDE: -To switch between branches of the current project repository, select the dropdown list -in the top of the sidebar to open a list of branches. -You must commit or discard all your changes before switching to a -different branch. +1. On the status bar, in the lower-left corner, select the current branch name. +1. From the dropdown list, select **Create new branch...**. +1. Enter the branch name. +1. Press <kbd>Enter</kbd>. -## Markdown editing +If you don't have write access to the repository, **Create new branch...** is not visible. -> - Support for pasting images [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22822) in GitLab 13.1. -> - Side-by-side Markdown preview [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68274) in GitLab 14.3. - -To edit Markdown files in the Web IDE: +## Commit changes -1. Go to your repository, and navigate to the Markdown page you want to edit. -1. Select **Open in Web IDE**, and GitLab loads the page in a tab in the editor. -1. Make your changes to the file. GitLab supports [GitLab Flavored Markdown (GLFM)](../../markdown.md). -1. When your changes are complete, select **Commit** in the left sidebar. -1. Add a commit message, select the branch you want to commit to, and select **Commit**. +To commit changes in the Web IDE: -When editing, you can upload local images by pasting them directly in the Markdown file. -The image is uploaded to the same directory and is named `image.png` by default. -If another file already exists with the same name, a numeric suffix is automatically -added to the filename. +1. On the activity bar on the left, select **Source Control**, + or press <kbd>Control</kbd>+<kbd>Shift</kbd>+<kbd>G</kbd>. +1. Enter your commit message. +1. Select **Commit & Push**. +1. Commit to the current branch, or create a new branch. -There are two ways to preview Markdown content in the Web IDE: +## Use the command palette -1. At the top of the file's tab, select **Preview Markdown** to preview the formatting - in your file. You can't edit the file in this view. - 1. To add more changes to the file, select **Edit**. -1. Right-click or use the keyboard shortcut `Command/Control + Shift + P` and - select **Preview Markdown** to toggle a live Markdown preview panel. +In the Web IDE, you can access many commands through the command palette. +To open the command palette and run a command in the Web IDE: -<!--- start_remove The following content will be removed on remove_date: '2023-02-01' --> +1. Press <kbd>Shift</kbd>+<kbd>Command</kbd>+<kbd>P</kbd>. +1. In the search box, start typing the command name. +1. From the dropdown list, select the command. -## Live Preview (removed) +## Edit settings -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108627) in GitLab 15.8 -and is planned for removal in 15.9. This change is a breaking change. +You can use the settings editor to view and modify your user and workspace settings. +To open the settings editor in the Web IDE: -> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/213853) from _Client Side Evaluation_ to _Live Preview_ in GitLab 13.0. +- On the top menu bar, select **File > Preferences > Settings**, + or press <kbd>Command</kbd>+<kbd>,</kbd>. -You can use the Web IDE to preview JavaScript projects right in the browser. -This feature uses CodeSandbox to compile and bundle the JavaScript used to -preview the web application. +In the settings editor, you can search for the settings you want to modify. -![Web IDE Live Preview](img/live_preview_v13_0.png) +## Edit keyboard shortcuts -Additionally, for public projects an **Open in CodeSandbox** button is available -to transfer the contents of the project into a public CodeSandbox project to -quickly share your project with others. +You can use the keyboard shortcuts editor to view and modify the default keybindings for all available commands. +To open the keyboard shortcuts editor in the Web IDE: -### Enable Live Preview +- On the top menu bar, select **File > Preferences > Keyboard Shortcuts**, + or press <kbd>Command</kbd>+<kbd>K</kbd> then <kbd>Command</kbd>+<kbd>S</kbd>. -With Live Preview enabled, you can preview projects with a `package.json` file and -a `main` entry point inside the Web IDE. +In the keyboard shortcuts editor, you can search for: -Live Preview is enabled for all projects on GitLab.com. If you are an administrator -of a self-managed GitLab instance, and you want to enable Live Preview: +- The keybindings you want to change +- The commands you want to add or remove keybindings for -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Scroll to **Web IDE** and select **Expand**: - ![Administrator Live Preview setting](img/admin_live_preview_v13_0.png) -1. Select **Enable Live Preview** and select **Save changes**. +Keybindings are based on your keyboard layout. If you change your keyboard layout, existing keybindings are updated automatically. -[In GitLab 12.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/268288), -third-party assets and libraries required for Live Preview are hosted at -`https://sandbox-prod.gitlab-static.net` when it is enabled. However, some libraries -are still served from other third-party services, which may or may not be desirable -in your environment. +## Change themes -An example `package.json`: +You can choose between different themes for the Web IDE. The default theme for the Web IDE is **GitLab Dark**. -```json -{ - "main": "index.js", - "dependencies": { - "vue": "latest" - } -} -``` +To change the Web IDE theme: -<!--- end_remove --> +1. On the top menu bar, select **File > Preferences > Theme > Color Theme**, + or press <kbd>Command</kbd>+<kbd>K</kbd> then <kbd>Command</kbd>+<kbd>T</kbd>. +1. From the dropdown list, preview the themes with the arrow keys. +1. Select a theme. -## Interactive Web Terminals for the Web IDE +The active color theme is stored in the [user settings](#edit-settings). -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211685) from GitLab Ultimate to GitLab Free in 13.1. +<!-- ## Privacy and data collection for extensions -WARNING: -Interactive Web Terminals for the Web IDE is currently in [**Beta**](../../../policy/alpha-beta-support.md#beta-features). -GitLab.com shared runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab/-/issues/24674), -so you must use your own private runner to make use of this feature. +The Web IDE Extension Marketplace is based on Open VSX. Open VSX does not collect any +data about you or your activities on the platform. -[Interactive Web Terminals](../../../ci/interactive_web_terminal/index.md) -give the project [Maintainers](../../permissions.md#project-members-permissions) -user access to a terminal to interact with the runner directly from -GitLab, including through the Web IDE. +However, the privacy and data collection practices of extensions available on Open VSX can vary. +Some extensions might collect data to provide personalized recommendations or to improve the functionality. +Other extensions might collect data for analytics or advertising purposes. -### Runner configuration +To protect your privacy and data: -Some things must be configured in the runner for the interactive web terminal -to work: +- Carefully review the permissions requested by an extension before you install the extension. +- Keep your extensions up to date to ensure that any security or privacy vulnerabilities are addressed promptly. --> -- The runner needs to have - [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section). - This section requires at least a `session_timeout` value (which defaults to 1800 - seconds) and a `listen_address` value. If `advertise_address` is not defined, `listen_address` is used. -- If you are using a reverse proxy with your GitLab instance, web terminals must be - [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). +## Interactive web terminals for the Web IDE (Beta) -If you have the terminal open and the job has finished with its tasks, the -terminal blocks the job from finishing for the duration configured in -[`[session_server].session_timeout`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) -until you close the terminal window. - -NOTE: -Not all executors are -[supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). -The [File Sync](#file-syncing-to-web-terminal) feature is supported on Kubernetes runners only. - -### Web IDE configuration file - -To enable the Web IDE terminals you must create the file -`.gitlab/.gitlab-webide.yml` inside the repository's root. This -file is fairly similar to the [`.gitlab-ci.yml` file](../../../ci/yaml/index.md) -syntax but with some restrictions: - -- No global blocks (such as `before_script` or `after_script`) can be defined. -- Only one job named `terminal` can be added to this file. -- Only the keywords `image`, `services`, `tags`, `before_script`, `script`, and - `variables` are allowed to be used to configure the job. -- To connect to the interactive terminal, the `terminal` job must be still alive - and running, otherwise the terminal cannot connect to the job's session. - By default the `script` keyword has the value `sleep 60` to prevent - the job from ending and giving the Web IDE enough time to connect. This means - that, if you override the default `script` value, you have to add a command - which would keep the job running, like `sleep`. - -For example, with this configuration file: - -```yaml -terminal: - # This can be any image that has the necessary runtime environment for your project. - image: node:10-alpine - before_script: - - apk update - script: sleep 60 - variables: - RAILS_ENV: "test" - NODE_ENV: "test" -``` - -After the terminal starts, the console is displayed and you can access -the project repository files. - -When you use the `image` keyword, a container with the specified image is created. -If you use the [shell executor](https://docs.gitlab.com/runner/executors/shell.html) -or the [SSH executor](https://docs.gitlab.com/runner/executors/ssh.html), `image` has no effect. - -The terminal job is branch dependent. The configuration file used to trigger -and configure the terminal is the one in the selected branch of the Web IDE. -If no configuration file exists in a branch, an error message is displayed. - -### Running interactive terminals in the Web IDE - -If Interactive Terminals are available for the current user, the **Terminal** button is visible in the right sidebar of the Web IDE. Select this button to open -or close the terminal tab. - -After opening, the tab shows the **Start Web Terminal** button. This button may -be disabled if the environment is not configured correctly. If so, a status -message describes the issue. Here are some reasons why **Start Web Terminal** -may be disabled: - -- `.gitlab/.gitlab-webide.yml` does not exist or is set up incorrectly. -- No active private runners are available for the project. - -If active, selecting the **Start Web Terminal** button loads the terminal view -and start connecting to the runner's terminal. At any time, the **Terminal** tab -can be closed and reopened and the state of the terminal is not affected. - -When the terminal is started and is successfully connected to the runner, then the -runner's shell prompt appears in the terminal. From here, you can enter -commands executed in the runner's environment. This is similar -to running commands in a local terminal or through SSH. - -While the terminal is running, it can be stopped by selecting **Stop Terminal**. -This disconnects the terminal and stops the runner's terminal job. From here, -select **Restart Terminal** to start a new terminal session. - -### File syncing to web terminal - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5276) in GitLab 12.0. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211686) from GitLab Ultimate to GitLab Free in 13.1. - -File changes in the Web IDE can be synced to a running web terminal. -This enables users to test their code changes in a preconfigured terminal -environment. - -NOTE: -Only file changes in the Web IDE are synced to the terminal. -Changes made in the terminal are **not** synced to the Web IDE. -This feature is only available for Kubernetes runners. - -To enable file syncing to the web terminal, the `.gitlab/.gitlab-webide.yml` -file needs to have a `webide-file-sync` service configured. Here is an example -configuration for a Node JS project which uses this service: - -```yaml -terminal: - # This can be any image that has the necessary runtime environment for your project. - image: - name: node:10-alpine - services: - - name: registry.gitlab.com/gitlab-org/webide-file-sync:latest - alias: webide-file-sync - entrypoint: ["/bin/sh"] - command: ["-c", "sleep 5 && ./webide-file-sync -project-dir $CI_PROJECT_DIR"] - ports: - # The `webide-file-sync` executable defaults to port 3000. - - number: 3000 -``` - -- The `webide-file-sync` executable must start **after** the project - directory is available. This is why we must add `sleep 5` to the `command`. - See [this issue](https://gitlab.com/gitlab-org/webide-file-sync/-/issues/7) for - more information. -- `$CI_PROJECT_DIR` is a - [predefined CI/CD variable](../../../ci/variables/predefined_variables.md) - for GitLab Runners. This is where your project's repository resides. - -After you have configured the web terminal for file syncing, then when the web -terminal is started, a **Terminal** status is visible in the status bar. - -![Web IDE Client Side Evaluation](img/terminal_status.png) - -Changes made to your files via the Web IDE sync to the running terminal -when: - -- <kbd>Control</kbd> + <kbd>S</kbd> (or <kbd>Command</kbd> + <kbd>S</kbd> on Mac) - is pressed while editing a file. -- You select any area outside the file editor after editing a file. -- A file or folder is created, deleted, or renamed. - -### Known issues - -The Web IDE has a few limitations: - -- Interactive Terminals is in a beta phase and continues to be improved in upcoming releases. In the meantime, the user is limited to having only one - active terminal at a time. -- LFS files can be rendered and displayed but they cannot be updated and committed using the Web IDE. If an LFS file is modified and pushed to the repository, the LFS pointer in the repository is overwritten with the modified LFS file content. +When you set up a remote development server in the Web IDE, you can use interactive web terminals to: -### Troubleshooting +- Access a remote shell on the server. +- Interact with the server's file system and execute commands remotely. -- If the terminal's text is gray and unresponsive, then the terminal has stopped - and it can no longer be used. A stopped terminal can be restarted by selecting - **Restart Terminal**. -- If the terminal displays **Connection Failure**, then the terminal could not - connect to the runner. Try to stop and restart the terminal. If the - problem persists, double check your runner configuration. - -## Related topics +You cannot use interactive web terminals to interact with a runner. +However, you can use a terminal to install dependencies and compile and debug code. -- [Web IDE Beta](../web_ide_beta/index.md) +For more information about configuring a workspace that supports interactive web terminals, see [remote development](../remote_development/index.md). diff --git a/doc/user/project/web_ide_beta/index.md b/doc/user/project/web_ide_beta/index.md index fe110baf578..a4c733be376 100644 --- a/doc/user/project/web_ide_beta/index.md +++ b/doc/user/project/web_ide_beta/index.md @@ -1,106 +1,11 @@ --- -stage: Create -group: Editor -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../web_ide/index.md' +remove_date: '2023-08-22' --- -# Web IDE Beta **(FREE)** +This document was moved to [another location](../web_ide/index.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.7 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. - -As announced in [this blog post](https://about.gitlab.com/blog/2022/05/23/the-future-of-the-gitlab-web-ide/), -the current implementation of the [Web IDE](../web_ide/index.md) is being replaced -with an implementation powered by Visual Studio Code. This effort is still under -development. For updates, see [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7683). - -To pair the Web IDE Beta with a Remote Development environment, see [Remote Development](../remote_development/index.md). - -## Enable the Web IDE Beta - -To use the Web IDE Beta on a self-managed GitLab instance, -ensure that the `vscode_web_ide` feature flag -[is enabled](../../../administration/feature_flags.md). - -On GitLab.com, this feature is available by default. However, you can -[stop using it if you choose](#stop-using-the-web-ide-beta). - -## Use the Web IDE Beta - -To open the Web IDE Beta from anywhere in the UI: - -- Use the <kbd>.</kbd> [keyboard shortcut](../../shortcuts.md). - -You can also open the Web IDE Beta when viewing a file, the repository file list, -or a merge request. - -### Use when viewing a file or the repository file list - -To open the Web IDE Beta from a file or the repository file list: - -- In the upper right of the page, select **Open in Web IDE**. - -If **Open in Web IDE** is not visible: - -1. Next to **Edit** or **Gitpod**, select the down arrow (**{chevron-lg-down}**). -1. From the list, select **Open in Web IDE**. -1. Select **Open in Web IDE**. - -### Use when viewing a merge request - -To open the Web IDE Beta from a merge request: - -1. Go to your merge request. -1. In the upper right corner, select **Code > Open in Web IDE**. - -## Open a file in the Web IDE Beta - -To open any file by its name: - -1. Press <kbd>Command</kbd>+<kbd>P</kbd>. -1. Enter the name of your file. - -![fuzzy_finder_v15_7](img/fuzzy_finder_v15_7.png) - -## Search across files - -You can use VS Code to quickly search all files in the opened folder. - -To search across files: - -1. Press <kbd>Shift</kbd>+<kbd>Command</kbd>+<kbd>F</kbd>. -1. Enter your search term. - -In the Web IDE Beta, only partial results from opened files are displayed. -Full file search is planned for a later date. - -## View a list of changed files - -To view a list of files you changed in the Web IDE Beta, -in the Activity Bar on the left, select **Source Control**. -Your `CHANGES`, `STAGED CHANGES`, and `MERGE CHANGES` are displayed. - -For details, see the [VS Code documentation](https://code.visualstudio.com/docs/sourcecontrol/overview#_commit). - -## Stop using the Web IDE Beta - -If you do not want to use the Web IDE Beta, you can change your personal preferences. - -1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. -1. In the **Web IDE** section, select the **Opt out of the Web IDE Beta** checkbox. -1. Select **Save changes**. - -## Known issues - -The [Web Terminal](../web_ide/index.md#interactive-web-terminals-for-the-web-ide) -and [Live Preview](../web_ide/index.md#live-preview-removed) are not available in the Web IDE Beta. - -These features might become available at a later date. - -## Related topics - -- [Remote Development](../remote_development/index.md) -- [Web IDE](../web_ide/index.md) +<!-- This redirect file can be deleted after <2023-08-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 53aaa41319b..9327ce53b3f 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -20,7 +20,7 @@ Group wikis are similar to [project wikis](index.md), with a few limitations: For updates, follow [the epic that tracks feature parity with project wikis](https://gitlab.com/groups/gitlab-org/-/epics/2782). Similar to project wikis, group members with at least the Developer role -and higher can edit group wikis. Group wiki repositories can be moved using the +can edit group wikis. Group wiki repositories can be moved using the [Group repository storage moves API](../../../api/group_repository_storage_moves.md). ## View a group wiki @@ -38,8 +38,8 @@ To access a group wiki: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53247) in GitLab 13.9. Users with the Owner role in a group can -[import and export group wikis](../../group/settings/import_export.md) when importing -or exporting a group. +[import or export a group wiki](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated) when they +import or export a group. Content created in a group wiki is not deleted when an account is downgraded or a GitLab trial ends. The group wiki data is exported whenever the group owner of @@ -48,8 +48,8 @@ the wiki is exported. To access the group wiki data from the export file if the feature is no longer available, you have to: -1. Extract the [export file tarball](../../group/settings/import_export.md) with - this command, replacing `FILENAME` with your file's name: +1. Extract the [export file tarball](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated) + with this command, replacing `FILENAME` with your file's name: `tar -xvzf FILENAME.tar.gz` 1. Browse to the `repositories` directory. This directory contains a [Git bundle](https://git-scm.com/docs/git-bundle) with the extension `.wiki.bundle`. @@ -71,7 +71,7 @@ To open group settings: 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. 1. Scroll to **Wiki** and select one of these options: - - **Enabled**: Everyone who can access the group can access the wiki. + - **Enabled**: For public groups, everyone can access the wiki. For internal groups, only authenticated users can access the wiki. - **Private**: Only group members can access the wiki. - **Disabled**: The wiki isn't accessible, and cannot be downloaded. 1. Select **Save changes**. diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index f01331ac784..eb13814f2ad 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Plan +group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index a0e6a78dfe2..69087791a3e 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- @@ -11,12 +11,9 @@ code are saved in projects, and most features are in the scope of projects. ## View projects -To view projects, on the top bar, select **Main menu > Projects > View all projects**. +To view all your projects, on the top bar, select **Main menu > Projects > View all projects**. -NOTE: -The **Explore projects** tab is visible to unauthenticated users unless the -[**Public** visibility level](../admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels) -is restricted. Then the tab is visible only to authenticated users. +To browse all public projects, select **Main menu > Explore > Projects**. ### Who can view the Projects page @@ -63,7 +60,7 @@ You can add a star to projects you use frequently to make them easier to find. To add a star to a project: 1. On the top bar, select **Main menu > Projects** and find your project. -1. In the upper right corner of the page, select **Star**. +1. In the upper-right corner of the page, select **Star**. ## View starred projects @@ -241,15 +238,15 @@ Configure Git to either: - Embed credentials in the request URL: - ```shell - git config --global url."https://${user}:${personal_access_token}@gitlab.example.com".insteadOf "https://gitlab.example.com" - ``` + ```shell + git config --global url."https://${user}:${personal_access_token}@gitlab.example.com".insteadOf "https://gitlab.example.com" + ``` - Use SSH instead of HTTPS: - ```shell - git config --global url."git@gitlab.example.com:".insteadOf "https://gitlab.example.com/" - ``` + ```shell + git config --global url."git@gitlab.example.com:".insteadOf "https://gitlab.example.com/" + ``` ### Disable Go module fetching for private projects @@ -263,8 +260,8 @@ the [environment variables](../../development/go_guide/dependencies.md#proxies): To disable fetching: 1. Disable `GOPRIVATE`: - - To disable queries for one project, disable `GOPRIVATE=gitlab.example.com/my/private/project`. - - To disable queries for all projects on GitLab.com, disable `GOPRIVATE=gitlab.example.com`. + - To disable queries for one project, disable `GOPRIVATE=gitlab.example.com/my/private/project`. + - To disable queries for all projects on GitLab.com, disable `GOPRIVATE=gitlab.example.com`. 1. Disable proxy queries in `GONOPROXY`. 1. Disable checksum queries in `GONOSUMDB`. @@ -291,8 +288,8 @@ To access the Geo secondary server with SSH: git config --global url."git@gitlab-secondary.example.com".insteadOf "http://gitlab.example.com" ``` - - For `gitlab.example.com`, use the primary site domain name. - - For `gitlab-secondary.example.com`, use the secondary site domain name. + - For `gitlab.example.com`, use the primary site domain name. + - For `gitlab-secondary.example.com`, use the secondary site domain name. 1. Ensure the client is set up for SSH access to GitLab repositories. You can test this on the primary, and GitLab replicates the public key to the secondary. @@ -331,7 +328,7 @@ download starts, the `insteadOf` configuration sends the traffic to the secondar - [Import a project](../../user/project/import/index.md). - [Connect an external repository to GitLab CI/CD](../../ci/ci_cd_for_external_repos/index.md). -- [Fork a project](repository/forking_workflow.md#creating-a-fork). +- [Fork a project](repository/forking_workflow.md#create-a-fork). - [Adjust project visibility and access levels](settings/index.md#configure-project-visibility-features-and-permissions). - [Limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names) diff --git a/doc/user/public_access.md b/doc/user/public_access.md index 2a9a9fb84fe..58eebc16d74 100644 --- a/doc/user/public_access.md +++ b/doc/user/public_access.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -95,8 +95,7 @@ For more information, see [Restrict visibility levels](admin_area/settings/visib ## Related topics -- For more granular control of project features, you can - [change the visibility of features](project/working_with_projects.md#change-the-visibility-of-individual-features-in-a-project). +- [Change the visibility of features](project/working_with_projects.md#change-the-visibility-of-individual-features-in-a-project) <!-- ## Troubleshooting diff --git a/doc/user/report_abuse.md b/doc/user/report_abuse.md index aabc9c5dff1..c4b9af28220 100644 --- a/doc/user/report_abuse.md +++ b/doc/user/report_abuse.md @@ -34,6 +34,8 @@ To report abuse from a user's profile page: ## Report abuse from a user's comment +> Reporting abuse from comments in epics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/389992) in GitLab 15.10. + To report abuse from a user's comment: 1. In the comment, in the upper-right corner, select **More actions** (**{ellipsis_v}**). @@ -48,7 +50,7 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Report abuse from an issue -1. On the issue, in the upper-right corner, select the vertical ellipsis (**{ellipsis_v}**). +1. On the issue, in the upper-right corner, select **Issue actions** (**{ellipsis_v}**). 1. Select **Report abuse to administrator**. 1. Select a reason for reporting the user. 1. Complete an abuse report. @@ -56,7 +58,7 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Report abuse from a merge request -1. On the merge request, in the upper-right corner, select the vertical ellipsis (**{ellipsis_v}**). +1. On the merge request, in the upper-right corner, select **Merge request actions** (**{ellipsis_v}**). 1. Select **Report abuse to administrator**. 1. Select a reason for reporting this user. 1. Complete an abuse report. @@ -64,5 +66,4 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Related topics -- Administrators can view and resolve abuse reports. - For more information, see [abuse reports administration documentation](admin_area/review_abuse_reports.md). +- [Abuse reports administration documentation](admin_area/review_abuse_reports.md) diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 372ec141112..6e8a7e7c1cf 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Organization +stage: Data Stores +group: Tenant Scale info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md index ed1d3b1d290..1444e5385f9 100644 --- a/doc/user/search/advanced_search.md +++ b/doc/user/search/advanced_search.md @@ -5,18 +5,18 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Advanced Search **(PREMIUM)** +# Advanced search **(PREMIUM)** > Moved to GitLab Premium in 13.9. -You can use Advanced Search for faster, more efficient search across the entire GitLab -instance. Advanced Search is based on Elasticsearch, a purpose-built full-text search +You can use advanced search for faster, more efficient search across the entire GitLab +instance. Advanced search is based on Elasticsearch, a purpose-built full-text search engine you can horizontally scale to get results in up to a second in most cases. You can find code you want to update in all projects at once to save maintenance time and promote innersourcing. -You can use Advanced Search in: +You can use advanced search in: - Projects - Issues @@ -29,37 +29,22 @@ You can use Advanced Search in: - Commits - Project wikis (not [group wikis](../project/wiki/group.md)) -For Advanced Search: +## Enable advanced search -- You can only search files smaller than 1 MB. - For self-managed GitLab instances, an administrator can - [change this limit](../../integration/advanced_search/elasticsearch.md#advanced-search-configuration). -- You can't use any of the following characters in the search query: - - ```plaintext - . , : ; / ` ' = ? $ & ^ | ~ < > ( ) { } [ ] @ - ``` - -- Only the default branch of a project is indexed for code search. - In a non-default branch, basic search is used. -- Search results show only the first match in a file, - but there might be more results in that file. - -## Enable Advanced Search - -- On GitLab.com, Advanced Search is enabled for groups with paid subscriptions. +- On GitLab.com, advanced search is enabled for groups with paid subscriptions. - For self-managed GitLab instances, an administrator must - [enable Advanced Search](../../integration/advanced_search/elasticsearch.md#enable-advanced-search). + [enable advanced search](../../integration/advanced_search/elasticsearch.md#enable-advanced-search). ## Syntax -Advanced Search uses [Elasticsearch syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#simple-query-string-syntax). The syntax supports both exact and fuzzy search queries. +Advanced search uses [Elasticsearch syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#simple-query-string-syntax). The syntax supports both exact and fuzzy search queries. <!-- markdownlint-disable --> | Syntax | Description | Example | |--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------| | `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) | +| `~` | Fuzzy search | [`J~ Doe`](https://gitlab.com/search?scope=users&search=j%7E+doe) | | <code>|</code> | Or | [<code>display | banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) | | `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) | | `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) | @@ -68,16 +53,23 @@ Advanced Search uses [Elasticsearch syntax](https://www.elastic.co/guide/en/elas | `#` | Issue ID | [`#23456`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964) | | `!` | Merge request ID | [`!23456`](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964) | +### Refining user search + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388409) in GitLab 15.10. + +In user search, a [fuzzy query](https://www.elastic.co/guide/en/elasticsearch/reference/7.2/query-dsl-fuzzy-query.html) is used by default. You can refine your search with [Elasticsearch syntax](#syntax). + ### Code search | Syntax | Description | Example | |--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------| | `filename:` | Filename | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) | -| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) | -| `extension:` | File extension without `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) | -| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) | +| `path:` | Repository location <sup>1</sup> | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) | +| `extension:` | File extension without `.` <sup>2</sup> | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) | +| `blob:` | Git object ID <sup>2</sup> | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) | -`extension:` and `blob:` return exact matches only. +1. `path:` returns matches for full paths or subpaths. +1. `extension:` and `blob:` return exact matches only. ### Examples @@ -87,5 +79,21 @@ Advanced Search uses [Elasticsearch syntax](https://www.elastic.co/guide/en/elas | [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Returns `RSpec.describe Resolvers` that does not start with `builder`. | | [<code>bug | (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Returns `bug` or both `display` and `banner`. | | [<code>helper -extension:yml -extension:js</code>](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=helper+-extension%3Ayml+-extension%3Ajs&snippets=) | Returns `helper` in all files except files with a `.yml` or `.js` extension. | +| [<code>helper path:lib/git</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=helper+path%3Alib%2Fgit) | Returns `helper` in all files with a `lib/git*` path (for example, `spec/lib/gitlab`). | + <!-- markdownlint-enable --> + +## Known issues + +- You can only search files smaller than 1 MB. + For self-managed GitLab instances, an administrator can + [change this limit](../../integration/advanced_search/elasticsearch.md#advanced-search-configuration). +- You can only use advanced search on the default branch of a project. +- The search query must not contain any of the following characters: + + ```plaintext + . , : ; / ` ' = ? $ & ^ | < > ( ) { } [ ] @ + ``` + +- Search results show only the first match in a file. diff --git a/doc/user/search/exact_code_search.md b/doc/user/search/exact_code_search.md index 14dca6d4a12..469d91b5194 100644 --- a/doc/user/search/exact_code_search.md +++ b/doc/user/search/exact_code_search.md @@ -7,12 +7,11 @@ type: reference # Exact Code Search **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/105049) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `index_code_with_zoekt` and `search_code_with_zoekt` which enables indexing and searching respectively. Both are disabled by default. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/105049) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `index_code_with_zoekt` for indexing and `search_code_with_zoekt` for searching. Both are disabled by default. WARNING: -Exact Code Search is an MVC. For the Exact Code Search feature roadmap, see [epic 9404](https://gitlab.com/groups/gitlab-org/-/epics/9404). -When this feature reaches the [**Alpha**](../../policy/alpha-beta-support.md#alpha-features) version, GitLab will dogfood it, and roll it out only to specific customers on GitLab.com who request access to it. -On self-managed GitLab, this feature is available and can be enabled. However, GitLab does not provide support or documentation at this development stage. +We are still actively making changes to the Exact Code Search feature. GitLab will dogfood it first, and roll it out only to specific customers on GitLab.com who request access to it. We will make an announcement when it's available for GitLab.com customers to tryout. You can follow our development progress by checking [the Exact Code Search feature roadmap](https://gitlab.com/groups/gitlab-org/-/epics/9404). +On self-managed GitLab, it is technically possible to enable this feature, however, GitLab does not provide support or documentation at this stage of development and it has not been widely tested at scale. There are also many known limitations. ## Usage @@ -20,9 +19,14 @@ When performing any Code search in GitLab it will choose to use "Exact Code Search" powered by [Zoekt](https://github.com/sourcegraph/zoekt) if the project is part of an enabled Group. -The main differences between Zoekt and [Advanced Search](advanced_search.md) +The main differences between Zoekt and [advanced search](advanced_search.md) are that Zoekt provides exact substring matching as well as allows you to search for regular expressions. Since it allows searching for regular expressions, certain special characters will require escaping. Backslash can escape special characters and wrapping in double quotes can be used for phrase searches. + +## Syntax + +To understand the possible filtering options, see the +[Zoekt query syntax](https://github.com/sourcegraph/zoekt/blob/main/doc/query_syntax.md). diff --git a/doc/user/search/global_search/advanced_search_syntax.md b/doc/user/search/global_search/advanced_search_syntax.md deleted file mode 100644 index c34c5c0c8fc..00000000000 --- a/doc/user/search/global_search/advanced_search_syntax.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../advanced_search.md' -remove_date: '2023-02-02' ---- - -This document was moved to [another location](../advanced_search.md). - -<!-- This redirect file can be deleted after <2023-02-02>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/search/index.md b/doc/user/search/index.md index dc07b4c9215..f6733abd305 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -50,7 +50,7 @@ Global search only flags with an error any search that includes more than: ## Perform a search -To start a search, type your search query in the search bar in the upper right of the screen. +To start a search, in the upper-right corner of the screen, in the search bar, type your search query. You must type at least two characters. ![search navbar](img/search_navbar_v15_7.png) @@ -82,16 +82,23 @@ where the results were found. 1. From the code search result, hover over the line number. 1. On the left, select **View blame**. - ![code search results](img/code_search_git_blame_v15_1.png) +![code search results](img/code_search_git_blame_v15_1.png) -## Search for projects by full path +### Filter code search results by language + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342651) in GitLab 15.10. + +To filter code search results by one or more languages: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. +1. On the code search page, on the left sidebar, select one or more languages. +1. On the left sidebar, select **Apply**. + +## Search for projects by full path -FLAG: -On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `full_path_project_search`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/388473) on GitLab.com in GitLab 15.9. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111808) on self-managed GitLab 15.10. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114932) in GitLab 15.11. Feature flag `full_path_project_search` removed. You can search for a project by entering its full path (including the namespace it belongs to) in the search box. As you type the project path, [autocomplete suggestions](#autocomplete-suggestions) are displayed. diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index d67c595512a..3b6c6105315 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -1,9 +1,8 @@ --- stage: Create -group: Editor +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/shortcuts.html' --- # GitLab keyboard shortcuts **(FREE)** @@ -15,7 +14,7 @@ To display a window in GitLab that lists its keyboard shortcuts, use one of the following methods: - Press <kbd>?</kbd>. -- In the Help menu in the upper right of the application, select **Keyboard shortcuts**. +- In the Help menu, in the upper-right corner of the application, select **Keyboard shortcuts**. Although [global shortcuts](#global-shortcuts) work from any area of GitLab, you must be in specific pages for the other shortcuts to be available, as @@ -69,7 +68,7 @@ relatively quickly to work, and they take you to another page in the project. | Keyboard shortcut | Description | |-----------------------------|-------------| -| <kbd>g</kbd> + <kbd>p</kbd> | Go to the project home page (**Project > Details**). | +| <kbd>g</kbd> + <kbd>o</kbd> | Go to the project overview page (**Project > Details**). | | <kbd>g</kbd> + <kbd>v</kbd> | Go to the project activity feed (**Project > Activity**). | | <kbd>g</kbd> + <kbd>r</kbd> | Go to the project releases list (**Project > Releases**). | | <kbd>g</kbd> + <kbd>f</kbd> | Go to the [project files](#project-files) list (**Repository > Files**). | @@ -81,8 +80,8 @@ relatively quickly to work, and they take you to another page in the project. | <kbd>i</kbd> | Go to the New Issue page (**Issues**, select **New Issue** ). | | <kbd>g</kbd> + <kbd>b</kbd> | Go to the project issue boards list (**Issues > Boards**). | | <kbd>g</kbd> + <kbd>m</kbd> | Go to the project [merge requests](project/merge_requests/index.md) list (**Merge Requests**). | +| <kbd>g</kbd> + <kbd>p</kbd> | Go to the CI/CD pipelines list (**CI/CD > Pipelines**). | | <kbd>g</kbd> + <kbd>j</kbd> | Go to the CI/CD jobs list (**CI/CD > Jobs**). | -| <kbd>g</kbd> + <kbd>l</kbd> | Go to the project metrics (**Monitor > Metrics**). | | <kbd>g</kbd> + <kbd>e</kbd> | Go to the project environments (**Deployments > Environments**). | | <kbd>g</kbd> + <kbd>k</kbd> | Go to the project Kubernetes cluster integration page (**Infrastructure > Kubernetes clusters**). You must have at least [`maintainer` permissions](permissions.md) to access this page. | | <kbd>g</kbd> + <kbd>s</kbd> | Go to the project snippets list (**Snippets**). | @@ -242,7 +241,7 @@ This shortcut is available when viewing a [wiki page](project/wiki/index.md): ### Content editor These shortcuts are available when editing a file with the -[Content Editor](https://about.gitlab.com/direction/create/editor/content_editor/): +[Content Editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/): | macOS shortcut | Windows shortcut | Description | |----------------|------------------|-------------| diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 0532ed27010..7ca897288e1 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -67,10 +67,10 @@ In GitLab versions 13.0 and later, snippets are [versioned by default](#versione To discover all snippets visible to you in GitLab, you can: - **View all snippets visible to you**: On the top bar of your GitLab - instance, select **Main menu > Snippets** to view your snippets dashboard. + instance, select **Main menu > Your work** and then **Snippets** to view your snippets dashboard. - **Visit [GitLab snippets](https://gitlab.com/dashboard/snippets)** for your snippets on GitLab.com. - **Explore all public snippets**: On the top bar of your GitLab - instance, select **Main menu > Snippets** and select **Explore snippets** to view + instance, select **Main menu > Explore** and select **Snippets** to view [all public snippets](https://gitlab.com/explore/snippets). - **View a project's snippets**: In your project, go to **Snippets**. diff --git a/doc/user/ssh.md b/doc/user/ssh.md index d44e6a0e071..b698f5a3edc 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -20,9 +20,7 @@ SSH uses two keys, a public key and a private key. - The public key can be distributed. - The private key should be protected. -When you need to copy or upload your SSH public key, make sure you do not accidentally copy or upload your private key instead. - -You cannot expose data by uploading your public key. When you need to copy or upload your SSH public key, make sure you do not accidentally copy or upload your private key instead. +It is not possible to reveal confidential data by uploading your public key. When you need to copy or upload your SSH public key, make sure you do not accidentally copy or upload your private key instead. You can use your private key to [sign commits](project/repository/ssh_signed_commits/index.md), which makes your use of GitLab and your data even more secure. @@ -208,7 +206,7 @@ the following command: ssh-keygen -o -t rsa -b 4096 -C "<comment>" ``` -## Generate an SSH key pair for a FIDO/U2F hardware security key +## Generate an SSH key pair for a FIDO2 hardware security key To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later: @@ -280,7 +278,7 @@ You can use [1Password](https://1password.com/) and the [1Password browser exten 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. 1. Select **Key**, and you should see the 1Password helper appear. 1. Select the 1Password icon and unlock 1Password. @@ -291,7 +289,7 @@ You can use [1Password](https://1password.com/) and the [1Password browser exten 1. Optional. Update **Expiration date** to modify the default expiration date. 1. Select **Add key**. -For more information about using 1Password with SSH keys, see the [1Password documentation](https://developer.1password.com/docs/ssh/get-started). +For more information about using 1Password with SSH keys, see the [1Password documentation](https://developer.1password.com/docs/ssh/get-started/). ## Add an SSH key to your GitLab account @@ -325,7 +323,7 @@ To use SSH with GitLab, copy your public key to your GitLab account: 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. 1. In the **Key** box, paste the contents of your public key. If you manually copied the key, make sure you copy the entire key, @@ -398,7 +396,7 @@ on `ssh` command options, see the `man` pages for both `ssh` and `ssh_config`. 1. Sign in to GitLab. 1. On the top bar, in the upper-right corner, select your avatar. -1. Select **Preferences**. +1. Select **Edit profile**. 1. On the left sidebar, select **SSH Keys**. Your existing SSH keys are listed at the bottom of the page. The information includes: @@ -548,7 +546,7 @@ If you receive this error, restart your terminal and try the command again. ### `Key enrollment failed: invalid format` error -You may receive the following error when [generating an SSH key pair for a FIDO/U2F hardware security key](#generate-an-ssh-key-pair-for-a-fidou2f-hardware-security-key): +You may receive the following error when [generating an SSH key pair for a FIDO2 hardware security key](#generate-an-ssh-key-pair-for-a-fido2-hardware-security-key): ```shell Key enrollment failed: invalid format @@ -557,7 +555,7 @@ Key enrollment failed: invalid format You can troubleshoot this by trying the following: - Run the `ssh-keygen` command using `sudo`. -- Verify your IDO/U2F hardware security key supports +- Verify your FIDO2 hardware security key supports the key type provided. - Verify the version of OpenSSH is 8.2 or greater by running `ssh -V`. diff --git a/doc/user/tasks.md b/doc/user/tasks.md index 0fc4c7571ab..fc232ee298e 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -158,6 +158,17 @@ To delete a task: 1. In the task window, in the options menu (**{ellipsis_v}**), select **Delete task**. 1. Select **OK**. +## Reorder tasks + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385887) in GitLab 16.0. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +By default, tasks are ordered by creation date. +To reorder them, drag them around. + ## Assign users to a task > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334810) in GitLab 15.4. @@ -165,7 +176,7 @@ To delete a task: To show who is responsible for a task, you can assign users to it. Users on GitLab Free can assign one user per task. -Users on GitLab Premium and higher can assign multiple users to a single task. +Users on GitLab Premium and Ultimate can assign multiple users to a single task. See also [multiple assignees for issues](project/issues/multiple_assignees_for_issues.md). Prerequisites: @@ -289,3 +300,19 @@ To add a task to an iteration: The task window opens. 1. Next to **Iteration**, select **Add to iteration**. 1. From the dropdown list, select the iteration to be associated with the task. + +## View task system notes + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) to feature flag named `work_items_mvc` in GitLab 15.8. Disabled by default. +> - Changing activity sort order [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) in GitLab 15.8. +> - Filtering activity [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/389971) in GitLab 15.10. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 15.10. + +You can view all the system notes related to the task. By default they are sorted by **Oldest first**. +You can always change the sorting order to **Newest first**, which is remembered across sessions. +You can also filter activity by **Comments only** and **History only** in addition to the default **All activity** which is remembered across sessions. + +## Comments and threads + +You can add [comments](discussions/index.md) and reply to threads in tasks. diff --git a/doc/user/todos.md b/doc/user/todos.md index 50b60dc5b4b..8f67311b559 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -1,5 +1,4 @@ --- -disqus_identifier: 'https://docs.gitlab.com/ee/workflow/todos.html' stage: Plan group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments @@ -21,7 +20,7 @@ You can use the To-Do List to track [actions](#actions-that-create-to-do-items) To access your To-Do List: -On the top bar, in the upper right, select To-Do List (**{task-done}**). +On the top bar, in the upper-right corner, select the To-Do List (**{task-done}**). ## Search the To-Do List @@ -34,12 +33,13 @@ Also, you can sort them by [**Label priority**](project/labels.md#set-label-prio ## Actions that create to-do items Many to-do items are created automatically. -A to-do item is added to your To-Do List when: +Some of the actions that add a to-do item to your To-Do List: - An issue or merge request is assigned to you. +- A [merge request review](project/merge_requests/reviews/index.md) is requested. - You're [mentioned](discussions/index.md#mentions) in the description or comment of an issue, merge request, or epic. -- You are mentioned in a comment on a commit or design. +- You're mentioned in a comment on a commit or design. - The CI/CD pipeline for your merge request fails. - An open merge request cannot be merged due to conflict, and one of the following is true: @@ -81,6 +81,8 @@ When you enable this feature: ## Create a to-do item +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390549) in objectives, key results and, tasks in GitLab 16.0. + You can manually add an item to your To-Do List. 1. Go to your: @@ -90,11 +92,15 @@ You can manually add an item to your To-Do List. - [Epic](group/epics/index.md) - [Design](project/issues/design_management.md) - [Incident](../operations/incident_management/incidents.md) + - [Objective or key result](../user/okrs.md) + - [Task](tasks.md) -1. On the right sidebar, at the top, select **Add a to do**. +1. In the upper-right corner, select **Add a to do** (**{todo-add}**). ![Adding a to-do item from the issuable sidebar](img/todos_add_todo_sidebar_v14_1.png) + ![Adding a to-do item from the Objective and Key results](img/todos_add_okrs_v16_0.png) + ## Create a to-do item by mentioning someone You can create a to-do item by mentioning someone anywhere except for a code block. Mentioning a user many times in one message only creates one to-do item. @@ -122,7 +128,7 @@ corresponding to-do item as done. To-do items are marked as done if you: -- Add an award emoji to the description or comment. +- Add an emoji reaction to the description or comment. - Add or remove a label. - Change the assignee. - Change the milestone. @@ -150,15 +156,17 @@ You can manually mark a to-do item as done. There are two ways to do this: - In the To-Do List, to the right of the to-do item, select **Mark as done** (**{check}**). -- In the sidebar of an issue, merge request, or epic, select **Mark as done**. +- In the upper-right corner of the resource (for example, issue or merge request), select **Mark as done** (**{todo-done}**). ![Mark as done from the sidebar](img/todos_mark_done_sidebar_v14_1.png) + ![Mark as done from the Objectives and Key results](img/todos_mark_done_okrs_v16_0.png) + ## Mark all to-do items as done You can mark all your to-do items as done at the same time. -In the To-Do List, in the upper right, select **Mark all as done**. +In the To-Do List, in the upper-right corner, select **Mark all as done**. ## How a user's To-Do List is affected when their access changes diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 0740108a168..822f169c2a3 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -76,8 +76,8 @@ Depending on your role, to manage your transfer usage you can [reduce Container Projects on GitLab SaaS have a 10 GB storage limit on their Git repository and LFS storage. After namespace-level storage limits are applied, the project limit is removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both. -When a project's repository and LFS reaches the quota, the project is locked. -You cannot push changes to a locked project. To monitor the size of each +When a project's repository and LFS reaches the quota, the project is set to a read-only state. +You cannot push changes to a read-only project. To monitor the size of each repository in a namespace, including a breakdown for each project, [view storage usage](#view-storage-usage). To allow a project's repository and LFS to exceed the free quota you must purchase additional storage. For more details, see [Excess storage usage](#excess-storage-usage). @@ -85,45 +85,45 @@ you must purchase additional storage. For more details, see [Excess storage usag ### Excess storage usage Excess storage usage is the amount that a project's repository and LFS exceeds the [project storage limit](#project-storage-limit). If no -purchased storage is available the project is locked. You cannot push changes to a locked project. -To unlock a project you must [purchase more storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) -for the namespace. When the purchase is completed, locked projects are automatically unlocked. The +purchased storage is available the project is set to a read-only state. You cannot push changes to a read-only project. +To remove the read-only state you must [purchase more storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) +for the namespace. When the purchase is completed, read-only projects are automatically restored to their standard state. The amount of purchased storage available must always be greater than zero. The **Storage** tab of the **Usage Quotas** page warns you of the following: - Purchased storage available is running low. -- Projects that are at risk of being locked if purchased storage available is zero. -- Projects that are locked because purchased storage available is zero. Locked projects are +- Projects that are at risk of becoming read-only if purchased storage available is zero. +- Projects that are read-only because purchased storage available is zero. Read-only projects are marked with an information icon (**{information-o}**) beside their name. #### Excess storage example The following example describes an excess storage scenario for a namespace: -| Repository | Storage used | Excess storage | Quota | Status | -|------------|--------------|----------------|--------|-------------------| -| Red | 10 GB | 0 GB | 10 GB | Locked **{lock}** | -| Blue | 8 GB | 0 GB | 10 GB | Not locked | -| Green | 10 GB | 0 GB | 10 GB | Locked **{lock}** | -| Yellow | 2 GB | 0 GB | 10 GB | Not locked | -| **Totals** | **30 GB** | **0 GB** | - | - | +| Repository | Storage used | Excess storage | Quota | Status | +|------------|--------------|----------------|--------|----------------------| +| Red | 10 GB | 0 GB | 10 GB | Read-only **{lock}** | +| Blue | 8 GB | 0 GB | 10 GB | Not read-only | +| Green | 10 GB | 0 GB | 10 GB | Read-only **{lock}** | +| Yellow | 2 GB | 0 GB | 10 GB | Not read-only | +| **Totals** | **30 GB** | **0 GB** | - | - | -The Red and Green projects are locked because their repositories and LFS have reached the quota. In this +The Red and Green projects are read-only because their repositories and LFS have reached the quota. In this example, no additional storage has yet been purchased. -To unlock the Red and Green projects, 50 GB additional storage is purchased. +To remove the read-only state from the Red and Green projects, 50 GB additional storage is purchased. Assuming the Green and Red projects' repositories and LFS grow past the 10 GB quota, the purchased storage -available decreases. All projects remain unlocked because 40 GB purchased storage is available: +available decreases. All projects remain read-only because 40 GB purchased storage is available: 50 GB (purchased storage) - 10 GB (total excess storage used). | Repository | Storage used | Excess storage | Quota | Status | |------------|--------------|----------------|---------|-------------------| -| Red | 15 GB | 5 GB | 10 GB | Not locked | -| Blue | 14 GB | 4 GB | 10 GB | Not locked | -| Green | 11 GB | 1 GB | 10 GB | Not locked | -| Yellow | 5 GB | 0 GB | 10 GB | Not locked | +| Red | 15 GB | 5 GB | 10 GB | Not read-only | +| Blue | 14 GB | 4 GB | 10 GB | Not read-only | +| Green | 11 GB | 1 GB | 10 GB | Not read-only | +| Yellow | 5 GB | 0 GB | 10 GB | Not read-only | | **Totals** | **45 GB** | **10 GB** | - | - | ## Namespace storage limit diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md index e4e1edd6346..922ce8e1354 100644 --- a/doc/user/workspace/index.md +++ b/doc/user/workspace/index.md @@ -1,42 +1,158 @@ --- -stage: Manage -group: Organization +stage: Create +group: IDE info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Organization +# Workspaces (Beta) **(PREMIUM)** -DISCLAIMER: -This page contains information related to upcoming products, features, and functionality. -It is important to note that the information presented is for informational purposes only. -Please do not rely on this information for purchasing or planning purposes. -As with all projects, the items mentioned on this page are subject to change or delay. -The development, release, and timing of any products, features, or functionality remain at the -sole discretion of GitLab Inc. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112397) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `remote_development_feature_flag`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/391543) in GitLab 16.0. -NOTE: -Organization is in development. +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `remote_development_feature_flag`. On GitLab.com, this feature is available. The feature is not ready for production use. -Organization will be above the [top-level namespaces](../namespace/index.md) for you to manage -everything you do as a GitLab administrator, including: +WARNING: +This feature is in [Beta](../../policy/alpha-beta-support.md#beta) and subject to change without notice. To leave feedback, see the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/410031). -- Defining and applying settings to all of your groups, subgroups, and projects. -- Aggregating data from all your groups, subgroups, and projects. +A workspace is a virtual sandbox environment for your code in GitLab. You can use workspaces to create and manage isolated development environments for your GitLab projects. These environments ensure that different projects don't interfere with each other. -Our goal is to reach feature parity between SaaS and self-managed installations, with all -[Admin Area settings](/ee/user/admin_area/settings/index.md) moving to either: +Each workspace includes its own set of dependencies, libraries, and tools, which you can customize to meet the specific needs of each project. Workspaces use the AMD64 architecture. -- Groups. Available in the Organization, and subgroups. -- Hardware Controls. For functionality that does not apply to groups, Hardware Controls are only - applicable to self-managed installations. There is one Hardware Controls section per installation. +## Set up a workspace -For more information about the state of organization development, -see [epic 9265](https://gitlab.com/groups/gitlab-org/-/epics/9265). +### Prerequisites -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a video introduction to the new hierarchy concept for groups and projects for epics, see -[Consolidating groups and projects update (August 2021)](https://www.youtube.com/watch?v=fE74lsG_8yM). +- Set up a Kubernetes cluster that the GitLab agent for Kubernetes supports. See the [supported Kubernetes versions](../clusters/agent/index.md#gitlab-agent-for-kubernetes-supported-cluster-versions). +- Ensure autoscaling for the Kubernetes cluster is enabled. +- In the Kubernetes cluster, verify that a [default storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) is defined so that volumes can be dynamically provisioned for each workspace. +- In the Kubernetes cluster, install an Ingress controller of your choice (for example, `ingress-nginx`), and make that controller accessible over a domain. For example, point `*.workspaces.example.dev` and `workspaces.example.dev` to the load balancer exposed by the Ingress controller. +- In the Kubernetes cluster, [install `gitlab-workspaces-proxy`](https://gitlab.com/gitlab-org/remote-development/gitlab-workspaces-proxy#installation-instructions). +- In the Kubernetes cluster, [install the GitLab agent for Kubernetes](../clusters/agent/install/index.md). +- Configure remote development settings for the GitLab agent with this snippet: -## Related topics + ```yaml + remote_development: + enabled: true + dns_zone: "workspaces.example.dev" + ``` -- [Organization developer documentation](../../development/workspace/index.md) + Update `dns_zone` as needed. + +- In each public project you want to use this feature for, define a [devfile](#devfile). Ensure the container images used in the devfile support [arbitrary user IDs](#arbitrary-user-ids). + +### Create a workspace + +To create a workspace in GitLab: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. In the root directory of your project, create a file named `.devfile.yaml`. +1. On the left sidebar, select **Workspaces**. +1. In the upper right, select **New workspace**. +1. From the **Select project** dropdown list, select a project with a `.devfile.yaml` file. You can only create workspaces for public projects. +1. From the **Select cluster agent** dropdown list, select a cluster agent owned by the group the project belongs to. +1. In **Time before automatic termination**, enter the number of hours until the workspace automatically terminates. This timeout is a safety measure to prevent a workspace from consuming excessive resources or running indefinitely. +1. Select **Create workspace**. + +The workspace might take a few minutes to start. To access the workspace, under **Preview**, select the workspace link. +You also have access to the terminal and can install any necessary dependencies. + +## Devfile + +A devfile is a file that defines a development environment by specifying the necessary tools, languages, runtimes, and other components for a GitLab project. + +Workspaces have built-in support for devfiles. You can specify a devfile for your project in the GitLab configuration file. The devfile is used to automatically configure the development environment with the defined specifications. + +This way, you can create consistent and reproducible development environments regardless of the machine or platform you use. + +### Relevant schema properties + +GitLab only supports the `container` and `volume` components in [devfile 2.2.0](https://devfile.io/docs/2.2.0/devfile-schema). +Use the `container` component to define a container image as the execution environment for a devfile workspace. +You can specify the base image, dependencies, and other settings. + +Only these properties are relevant to the GitLab implementation of the `container` component: + +| Properties | Definition | +|----------------| ----------------------------------------------------------------------------------| +| `image` | Name of the container image to use for the workspace. | +| `memoryRequest`| Minimum amount of memory the container can use. | +| `memoryLimit` | Maximum amount of memory the container can use. | +| `cpuRequest` | Minimum amount of CPU the container can use. | +| `cpuLimit` | Maximum amount of CPU the container can use. | +| `env` | Environment variables to use in the container. | +| `endpoints` | Port mappings to expose from the container. | +| `volumeMounts` | Storage volume to mount in the container. | + +### Example definition + +The following is an example devfile: + +```yaml +schemaVersion: 2.2.0 +components: + - name: tooling-container + attributes: + gl/inject-editor: true + container: + image: registry.gitlab.com/gitlab-org/remote-development/gitlab-remote-development-docs/debian-bullseye-ruby-3.2-node-18.12:rubygems-3.4-git-2.33-lfs-2.9-yarn-1.22-graphicsmagick-1.3.36-gitlab-workspaces + env: + - name: KEY + value: VALUE + endpoints: + - name: http-3000 + targetPort: 3000 +``` + +For more information, see the [devfile documentation](https://devfile.io/docs/2.2.0/devfile-schema). +For other examples, see the [`examples` projects](https://gitlab.com/gitlab-org/remote-development/examples). + +This container image is for demonstration purposes only. To use your own container image, see [Arbitrary user IDs](#arbitrary-user-ids). + +## Web IDE + +Workspaces are bundled with the Web IDE by default. The Web IDE is the only code editor available for workspaces. + +The Web IDE is powered by the [GitLab VS Code fork](https://gitlab.com/gitlab-org/gitlab-web-ide-vscode-fork). For more information, see [Web IDE](../project/web_ide/index.md). + +## Private repositories + +You cannot create a workspace for a private repository because GitLab does not inject any credentials into the workspace. You can only create a workspace for public repositories that have a devfile. + +From a workspace, you can clone any repository manually. + +## Pod interaction in a cluster + +Workspaces run as pods in a Kubernetes cluster. GitLab does not impose any restrictions on the manner in which pods interact with each other. + +Because of this requirement, you might want to isolate this feature from other containers in your cluster. + +## Network access and workspace authorization + +It's the client's responsibility to restrict network access to the Kubernetes control plane as GitLab does not have control over the API. + +Only the workspace creator can access the workspace and any endpoints exposed in that workspace. The workspace creator is only authorized to access the workspace after user authentication with OAuth. + +## Compute resources and volume storage + +When you stop a workspace, the compute resources for that workspace are scaled down to zero. However, the volume provisioned for the workspace still exists. + +To delete the provisioned volume, you must terminate the workspace. + +## Disable remote development in the GitLab agent for Kubernetes + +You can stop the `remote_development` module of the GitLab agent for Kubernetes from communicating with GitLab. To disable remote development in the GitLab agent configuration, set this property: + +```yaml +remote_development: + enabled: false +``` + +If you already have running workspaces, an administrator must manually delete these workspaces in Kubernetes. + +## Arbitrary user IDs + +You can provide your own container image, which can run as any Linux user ID. It's not possible for GitLab to predict the Linux user ID for a container image. +GitLab uses the Linux root group ID permission to create, update, or delete files in the container. CRI-O, the container runtime interface used by Kubernetes, has a default group ID of `0` for all containers. + +If you have a container image that does not support arbitrary user IDs, you cannot create, update, or delete files in a workspace. To create a container image that supports arbitrary user IDs, see the [OpenShift documentation](https://docs.openshift.com/container-platform/4.12/openshift_images/create-images.html#use-uid_create-images). |