diff options
Diffstat (limited to 'doc/user')
275 files changed, 4521 insertions, 2860 deletions
diff --git a/doc/user/admin_area/analytics/dev_ops_report.md b/doc/user/admin_area/analytics/dev_ops_report.md index 6ca5e5034bf..89d8a5ea054 100644 --- a/doc/user/admin_area/analytics/dev_ops_report.md +++ b/doc/user/admin_area/analytics/dev_ops_report.md @@ -20,46 +20,49 @@ To see DevOps Report: ## DevOps Score NOTE: -Your GitLab instance's [usage ping](../settings/usage_statistics.md#usage-ping) must be activated in order to use this feature. +To see the DevOps score, you must activate your GitLab instance's [Service Ping](../settings/usage_statistics.md#service-ping). -The DevOps Score tab displays the usage of major GitLab features on your instance over -the last 30 days, averaged over the number of billable users in that time period. It also -provides a Lead score per feature, which is calculated based on GitLab analysis -of top-performing instances based on [usage ping data](../settings/usage_statistics.md#usage-ping) that GitLab has -collected. Your score is compared to the lead score of each feature and then expressed as a percentage at the bottom of said feature. -Your overall **DevOps Score** is an average of your feature scores. You can use this score to compare your DevOps status to other organizations. - -The page also provides helpful links to articles and GitLab docs, to help you -improve your scores. +You can use the DevOps score to compare your DevOps status to other organizations. -Usage ping data is aggregated on GitLab servers for analysis. Your usage -information is **not sent** to any other GitLab instances. If you have just started using GitLab, it may take a few weeks for data to be -collected before this feature is available. +The DevOps Score tab displays the usage of major GitLab features on your instance over +the last 30 days, averaged over the number of billable users in that time period. +You can also see the Leader usage score, calculated from top-performing instances based on +[Service Ping data](../settings/usage_statistics.md#service-ping) that GitLab has collected. +Your score is compared to the lead score of each feature and then expressed +as a percentage at the bottom of said feature. Your overall **DevOps Score** is an average of your +feature scores. + +Service Ping data is aggregated on GitLab servers for analysis. Your usage +information is **not sent** to any other GitLab instances. +If you have just started using GitLab, it might take a few weeks for data to be collected before this +feature is available. ## DevOps Adoption **(ULTIMATE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta). > - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59267) in GitLab 14.0. > - Enabled on GitLab.com. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-or-enable-devops-adoption). **(ULTIMATE SELF)** +> - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. +> - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. -The DevOps Adoption tab shows you which groups within your organization are using the most essential features of GitLab: +DevOps Adoption shows you which groups in your organization are using the most essential features of GitLab: - Dev - - Issues - - Merge Requests - Approvals - Code owners + - Issues + - Merge requests - Sec - - Scans + - DAST + - SAST - Ops - - Runners - - Pipelines - Deployments + - Pipelines + - Runners -When managing groups in the UI, you can add your groups with the **Add group to table** -button, in the top right hand section the page. +To add your groups, in the top right-hand section the page, select **Add group to table**. DevOps Adoption allows you to: @@ -67,7 +70,7 @@ DevOps Adoption allows you to: - Identify specific groups that are lagging in their adoption of GitLab so you can help them along in their DevOps journey. - Find the groups that have adopted certain features and can provide guidance to other groups on how to use those features. - + ### Disable or enable DevOps Adoption diff --git a/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_0.png b/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_0.png Binary files differdeleted file mode 100644 index f4170b2938c..00000000000 --- a/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_0.png +++ /dev/null diff --git a/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_1.png b/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_1.png Binary files differnew file mode 100644 index 00000000000..79481e43e8e --- /dev/null +++ b/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_1.png diff --git a/doc/user/admin_area/analytics/user_cohorts.md b/doc/user/admin_area/analytics/user_cohorts.md deleted file mode 100644 index b906f9b8fa6..00000000000 --- a/doc/user/admin_area/analytics/user_cohorts.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../user_cohorts.md' -remove_date: '2021-06-01' ---- - -This document was moved to [another location](../user_cohorts.md). - -<!-- This redirect file can be deleted after <2021-06-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/admin_area/approving_users.md b/doc/user/admin_area/approving_users.md index 3d9722035d5..2852f73ffc8 100644 --- a/doc/user/admin_area/approving_users.md +++ b/doc/user/admin_area/approving_users.md @@ -47,7 +47,8 @@ To approve or reject a user sign up: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Pending approval** tab. -1. In the user's row, select settings (**{settings}**). +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. 1. Select **Approve** or **Reject**. Approving a user: diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index dfb37cb8646..8c5ae2dfb47 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -38,7 +38,7 @@ The following is an example of the Credentials inventory page: If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: -| Token state | [Token expiration enforced?](settings/account_and_limit_settings.md#do-not-enforce-personal-access-token-expiration) | Show Revoke button? | Comments | +| Token state | [Token expiration enforced?](settings/account_and_limit_settings.md#allow-expired-personal-access-tokens-to-be-used) | Show Revoke button? | Comments | |-------------|------------------------|--------------------|----------------------------------------------------------------------------| | Active | Yes | Yes | Allows administrators to revoke the PAT, such as for a compromised account | | Active | No | Yes | Allows administrators to revoke the PAT, such as for a compromised account | diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md index 6cf3c5bbd7d..12d143b3a13 100644 --- a/doc/user/admin_area/custom_project_templates.md +++ b/doc/user/admin_area/custom_project_templates.md @@ -9,36 +9,33 @@ type: reference > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6860) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.2. -GitLab administrators can configure the group where all the custom project -templates are sourced. +GitLab administrators can set a group to be the source of project templates that are +selectable when a new project is created on the instance. These templates can be selected +when you go to **New project > Create from template** and select the **Instance** tab. -Every project directly under the group namespace will be -available to the user if they have access to them. For example: +Every project in the group, but not its subgroups, can be selected when a new project +is created, based on the user's access permissions: -- Public projects, in the group will be available to every signed-in user, if all enabled [project features](../project/settings/index.md#sharing-and-permissions) +- Public projects can be selected by any signed-in user as a template for a new project, + if all enabled [project features](../project/settings/index.md#sharing-and-permissions) except for GitLab Pages are set to **Everyone With Access**. -- Private projects will be available only if the user is a member of the project. +- Private projects can be selected only by users who are members of the projects. Repository and database information that are copied over to each new project are -identical to the data exported with the -[GitLab Project Import/Export](../project/settings/import_export.md). +identical to the data exported with the [GitLab Project Import/Export](../project/settings/import_export.md). -NOTE: -To set project templates at a group level, -see [Custom group-level project templates](../group/custom_project_templates.md). +To set project templates at the group level, see [Custom group-level project templates](../group/custom_project_templates.md). -## Configuring +## Select instance-level project template group -GitLab administrators can configure a GitLab group that serves as template -source for an entire GitLab instance: +To select the group to use as the source for the project templates: 1. On the top bar, navigate to **Menu > Admin > Settings > Templates**. 1. Expand **Custom project templates**. 1. Select a group to use. 1. Select **Save changes**. -NOTE: -Projects below subgroups of the template group are **not** supported. +Projects in subgroups of the template group are **not** included in the template list. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md index 37fdb3ae195..4be1ace10aa 100644 --- a/doc/user/admin_area/diff_limits.md +++ b/doc/user/admin_area/diff_limits.md @@ -21,10 +21,11 @@ consumption of your instance. Keep this in mind when adjusting the maximum. To speed the loading time of merge request views and branch comparison views on your instance, you can configure three instance-level maximum values for diffs: -- **Maximum diff patch size**: The total size, in bytes, of the entire diff. -- **Maximum diff files**: The total number of files changed in a diff. -- **Maximum diff files**: The total number of files changed in a diff. The default value is 1000. -- **Maximum diff lines**: The total number of lines changed in a diff. The default value is 50,000. +| Value | Definition | Default value | Maximum value | +| ----- | ---------- | :-----------: | :-----------: | +| **Maximum diff patch size** | The total size, in bytes, of the entire diff. | 200 KB | 500 KB | +| **Maximum diff files** | The total number of files changed in a diff. | 1000 | 3000 | +| **Maximum diff lines** | The total number of lines changed in a diff. | 50,000 | 100,000 | When a diff reaches 10% of any of these values, the files are shown in a collapsed view, with a link to expand the diff. Diffs that exceed any of the @@ -35,7 +36,7 @@ To configure these values: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. In the left sidebar, select **Settings > General**. 1. Expand **Diff limits**. -1. Enter a value for **Maximum diff patch size**, measured in bytes. +1. Enter a value for the diff limit. 1. Select **Save changes**. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/img/index_runners_search_or_filter.png b/doc/user/admin_area/img/index_runners_search_or_filter.png Binary files differdeleted file mode 100644 index 5176a1a39bf..00000000000 --- a/doc/user/admin_area/img/index_runners_search_or_filter.png +++ /dev/null diff --git a/doc/user/admin_area/img/index_runners_search_or_filter_v14_1.png b/doc/user/admin_area/img/index_runners_search_or_filter_v14_1.png Binary files differnew file mode 100644 index 00000000000..ab196a0ca9e --- /dev/null +++ b/doc/user/admin_area/img/index_runners_search_or_filter_v14_1.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 2e4a8261c63..31fd1a3a0a1 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -160,7 +160,7 @@ You can impersonate a user in the following ways: 1. In the left sidebar, select **Overview > Users**. 1. From the list of users, select a user. 1. Select **Impersonate**. -- With the API, using [impersonation tokens](../../api/README.md#impersonation-tokens). +- With the API, using [impersonation tokens](../../api/index.md#impersonation-tokens). All impersonation activities are [captured with audit events](../../administration/audit_events.md#impersonation-data). @@ -276,21 +276,22 @@ To search runners' descriptions: You can also filter runners by status, type, and tag. To filter: 1. Click in the **Search or filter results...** field. -1. Select **status:**, **type:**, or **tag:**. +1. Select **Status**, **Type**, or **Tags**. 1. Select or enter your search criteria. - + For each runner, the following attributes are listed: | Attribute | Description | |--------------|-------------| -| Type | One or more of the following states: shared, group, specific, locked, or paused | -| Runner token | Token used to identify the runner, and which the runner uses to communicate with the GitLab instance | -| Description | Description given to the runner when it was created | +| Type/State | One or more of the following states: shared, group, specific, locked, or paused | +| Runner token | Partial token used to identify the runner, and which the runner uses to communicate with the GitLab instance | +| Runner ID | Numerical ID of the runner | +| Description | Description given to the runner | | Version | GitLab Runner version | | IP address | IP address of the host on which the runner is registered | -| Projects | Projects to which the runner is assigned | +| Projects | Number of projects to which the runner is assigned | | Jobs | Total of jobs run by the runner | | Tags | Tags associated with the runner | | Last contact | Timestamp indicating when the runner last contacted the GitLab instance | diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 58876b87576..57f643b75c7 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -107,7 +107,7 @@ For GitLab self-managed instances, you have a 14-day grace period before this occurs. - To resume functionality, upload a new license. -- To fall back to Free features, delete the expired license. +- To fall back to Free features, delete all expired licenses. ### Remove a license @@ -117,6 +117,8 @@ To remove a license from a self-managed instance: 1. On the left sidebar, select **License**. 1. Select **Remove license**. +These steps may need to be repeated to completely remove all licenses, including those applied in the past. + ## License history You can upload and view more than one license, but only the latest license in the current date diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index 71e72cc630c..3889dd93d59 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -23,8 +23,9 @@ or directly from the Admin Area. To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. -1. Select a user. -1. Under the **Account** tab, select **Block user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Block**. A blocked user: @@ -47,8 +48,9 @@ A blocked user can be unblocked from the Admin Area. To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select on the **Blocked** tab. -1. Select a user. -1. Under the **Account** tab, select **Unblock user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Unblock**. Users can also be unblocked using the [GitLab API](../../api/users.md#unblock-user). @@ -85,8 +87,9 @@ A user can be deactivated from the Admin Area. To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. -1. Select a user. -1. Under the **Account** tab, select **Deactivate user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Deactivate**. Please note that for the deactivation option to be visible to an admin, the user: @@ -126,8 +129,9 @@ To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Deactivated** tab. -1. Select a user. -1. Under the **Account** tab, select **Activate user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Activate**. Users can also be activated using the [GitLab API](../../api/users.md#activate-user). @@ -145,7 +149,7 @@ A deactivated user can also activate their account themselves by logging back in GitLab administrators can ban users. NOTE: -This feature is behind a feature flag that is disabled by default. GitLab administrators +This feature is behind a feature flag that is disabled by default. GitLab administrators with access to the GitLab Rails console can [enable](../../administration/feature_flags.md) this feature for your GitLab instance. @@ -157,8 +161,9 @@ Users can be banned using the Admin Area. To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. -1. Select a user. -1. Under the **Account** tab, select **Ban user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Ban user**. NOTE: This feature is a work in progress. Currently, banning a user @@ -172,8 +177,9 @@ A banned user can be unbanned using the Admin Area. To do this: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Select the **Banned** tab. -1. Select a user. -1. Under the **Account** tab, select **Unban user**. +1. (Optional) Select a user. +1. Select the **{settings}** **User administration** dropdown. +1. Select **Unban user**. NOTE: Unbanning a user changes the user's state to active and consumes a diff --git a/doc/user/admin_area/monitoring/background_migrations.md b/doc/user/admin_area/monitoring/background_migrations.md index 50593959710..cbaa4b30cb7 100644 --- a/doc/user/admin_area/monitoring/background_migrations.md +++ b/doc/user/admin_area/monitoring/background_migrations.md @@ -4,7 +4,7 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Batched Background Migrations **(FREE SELF)** +# Batched background migrations **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/51332) in GitLab 13.11. > - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. @@ -21,16 +21,19 @@ are created by GitLab developers and run automatically on upgrade. However, such limited in scope to help with migrating some `integer` database columns to `bigint`. This is needed to prevent integer overflow for some tables. -All migrations must be finished before upgrading GitLab. To check the status of the existing -migrations, execute this command: +## Check the status of background migrations **(FREE SELF)** -```ruby -Gitlab::Database::BackgroundMigration::BatchedMigration.pluck(:id, :table_name, :status) -``` +All migrations must have a `Finished` status before updating GitLab. To check the status of the existing +migrations: + +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Monitoring > Background Migrations**. + +  -## Enable or disable Batched Background Migrations **(FREE SELF)** +## Enable or disable batched background migrations **(FREE SELF)** -Batched Background Migrations is under development but ready for production use. +Batched background migrations are under development but ready for production use. It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to disable it. @@ -63,7 +66,7 @@ To maximize throughput of batched background migrations (in terms of the number ## Enable or disable automatic batch size optimization **(FREE SELF)** -Automatic batch size optimization for Batched Background Migrations is under development but ready for production use. +Automatic batch size optimization for batched background migrations is under development but ready for production use. It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to disable it. @@ -79,3 +82,27 @@ To disable it: ```ruby Feature.disable(:optimize_batched_migrations) ``` + +## Troubleshooting + +### Database migrations failing because of batched background migration not finished + +When updating to GitLab 14.2 or later there might be a database migration failing with a message like: + +```plaintext +StandardError: An error has occurred, all later migrations canceled: + +Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': + {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"push_event_payloads", :column_name=>"event_id", :job_arguments=>[["event_id"], ["event_id_convert_to_bigint"]]} +``` + +To fix this error: + +1. Update to either 14.0.5 or 14.1. +1. [Check the status](#check-the-status-of-background-migrations) of the batched background migration from the error message, and make sure it is listed as finished. If it is still active, either wait until it is done, or finalize it manually using the command suggested in the error, for example: + +```shell +sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,push_event_payloads,event_id,'[["event_id"]\, ["event_id_convert_to_bigint"]]'] +``` + +1. You can now update to GitLab 14.2 or higher. diff --git a/doc/user/admin_area/monitoring/img/batched_background_migrations_queued_v14_0.png b/doc/user/admin_area/monitoring/img/batched_background_migrations_queued_v14_0.png Binary files differnew file mode 100644 index 00000000000..0b0792b5e7a --- /dev/null +++ b/doc/user/admin_area/monitoring/img/batched_background_migrations_queued_v14_0.png diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md index a179eb70453..7816d0648b2 100644 --- a/doc/user/admin_area/review_abuse_reports.md +++ b/doc/user/admin_area/review_abuse_reports.md @@ -14,7 +14,7 @@ reports in the Admin Area. ## Receiving notifications of abuse reports -To receive notifications of new abuse reports by e-mail, follow these steps: +To receive notifications of new abuse reports by email, follow these steps: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Settings > Reporting**. diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index f2a849e1894..2b60ed5345b 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -199,7 +199,7 @@ Once a lifetime for personal access tokens is set, GitLab: allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, or remove it, before revocation takes place. -## Enforce SSH key expiration **(ULTIMATE SELF)** +## Allow expired SSH keys to be used **(ULTIMATE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250480) in GitLab 13.9. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/320970) in GitLab 14.0. @@ -215,15 +215,14 @@ To allow the use of expired SSH keys: Disabling SSH key expiration immediately enables all expired SSH keys. -## Do not enforce Personal Access Token expiration **(ULTIMATE SELF)** +## Allow expired Personal Access Tokens to be used **(ULTIMATE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214723) in GitLab Ultimate 13.1. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/296881) in GitLab 13.9. -By default, expired personal access tokens (PATs) cannot be used. -You can allow the use of expired PATs with the following steps: +By default, expired personal access tokens (PATs) **are not usable**. -To do this: +To allow the use of expired PATs: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. In the left sidebar, select **Settings > General**. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index ffe969a6799..69d86259409 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -88,7 +88,7 @@ The setting at all levels is only available to GitLab administrators. The default expiration time of the [job artifacts](../../../administration/job_artifacts.md) can be set in the Admin Area of your GitLab instance. The syntax of duration is -described in [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) +described in [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) and the default value is `30 days`. 1. On the top bar, select **Menu >** **{admin}** **Admin**. @@ -97,7 +97,7 @@ and the default value is `30 days`. 1. Click **Save changes** for the changes to take effect. This setting is set per job and can be overridden in -[`.gitlab-ci.yml`](../../../ci/yaml/README.md#artifactsexpire_in). +[`.gitlab-ci.yml`](../../../ci/yaml/index.md#artifactsexpire_in). To disable the expiration, set it to `0`. The default unit is in seconds. NOTE: @@ -195,8 +195,8 @@ As of June 22, 2020 the [value is set](../../gitlab_com/index.md#gitlab-cicd) to ## Protect CI/CD variables by default -To set all new [CI/CD variables](../../../ci/variables/README.md) as -[protected](../../../ci/variables/README.md#protect-a-cicd-variable) by default: +To set all new [CI/CD variables](../../../ci/variables/index.md) as +[protected](../../../ci/variables/index.md#protect-a-cicd-variable) by default: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Settings > CI/CD**. @@ -214,7 +214,7 @@ of your GitLab instance (`.gitlab-ci.yml` if not set): 1. Input the new file and path in the **Default CI/CD configuration file** field. 1. Hit **Save changes** for the changes to take effect. -It is also possible to specify a [custom CI/CD configuration file for a specific project](../../../ci/pipelines/settings.md#custom-cicd-configuration-file). +It is also possible to specify a [custom CI/CD configuration file for a specific project](../../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file). ## Required pipeline configuration **(PREMIUM SELF)** @@ -224,7 +224,7 @@ This feature is being re-evaluated in favor of a different We recommend that users who haven't yet implemented this feature wait for the new solution. -You can set a [CI/CD template](../../../ci/examples/README.md#cicd-templates) +You can set a [CI/CD template](../../../ci/examples/index.md#cicd-templates) as a required pipeline configuration for all projects on a GitLab instance. You can use a template from: @@ -233,13 +233,13 @@ use a template from: NOTE: When you use a configuration defined in an instance template repository, - nested [`include:`](../../../ci/yaml/README.md#include) keywords + nested [`include:`](../../../ci/yaml/index.md#include) keywords (including `include:file`, `include:local`, `include:remote`, and `include:template`) [do not work](https://gitlab.com/gitlab-org/gitlab/-/issues/35345). The project CI/CD configuration merges into the required pipeline configuration when a pipeline runs. The merged configuration is the same as if the required pipeline configuration -added the project configuration with the [`include` keyword](../../../ci/yaml/README.md#include). +added the project configuration with the [`include` keyword](../../../ci/yaml/index.md#include). To view a project's full merged configuration, [View the merged YAML](../../../ci/pipeline_editor/index.md#view-expanded-configuration) in the pipeline editor. @@ -280,6 +280,45 @@ To set the maximum file size: 1. Enter the maximum file size, in bytes. 1. Click **Save size limits**. +## Runner registration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22225) in GitLab 14.1. +> - [Deployed behind a feature flag](../../feature_flags.md), disabled by default. +> - Disabled on GitLab.com. +> - Not recommended for production use. +> - To use in GitLab self-managed instances, ask a GitLab administrator to enable it. **(FREE SELF)** + +GitLab administrators can adjust who is allowed to register runners, by showing and hiding areas of the UI. + +By default, all members of a project and group are able to register runners. + +To change this: + +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. Go to **Settings > CI/CD**. +1. Expand the **Runner registration** section. +1. Select the desired options. +1. Click **Save changes**. + +When the registration sections are hidden in the UI, members of the project or group that need to register runners must contact the administrators. + +This feature is currently behind a feature flag. +To enable it: + +**In Omnibus installations:** + +1. Enter the Rails console: + + ```shell + sudo gitlab-rails console + ``` + +1. Flip the switch and enable the feature flag: + + ```ruby + Feature.enable(:runner_registration_control) + ``` + ## Troubleshooting ### 413 Request Entity Too Large diff --git a/doc/user/admin_area/settings/gitaly_timeouts.md b/doc/user/admin_area/settings/gitaly_timeouts.md index 6f488efee11..04887906c91 100644 --- a/doc/user/admin_area/settings/gitaly_timeouts.md +++ b/doc/user/admin_area/settings/gitaly_timeouts.md @@ -8,23 +8,20 @@ type: reference # Gitaly timeouts **(FREE SELF)** [Gitaly](../../../administration/gitaly/index.md) timeouts are configurable. The timeouts can be -configured to make sure that long running Gitaly calls don't needlessly take up resources. +configured to make sure that long-running Gitaly calls don't needlessly take up resources. To access Gitaly timeout settings: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Settings > Preferences**. -1. Expand the **Gitaly** section. +1. Expand the **Gitaly timeouts** section. ## Available timeouts -The following timeouts can be modified: +The following timeouts are available. -- **Default Timeout Period**. This timeout is the default for most Gitaly calls. It should be shorter than the - worker timeout that can be configured for [Puma](https://docs.gitlab.com/omnibus/settings/puma.html#puma-settings). - Used to make sure that Gitaly calls made within a web request cannot exceed the entire request timeout. - Defaults to 55 seconds. - -- **Fast Timeout Period**. This is the timeout for very short Gitaly calls. Defaults to 10 seconds. -- **Medium Timeout Period**. This timeout should be between the default and the fast timeout. - Defaults to 30 seconds. +| Timeout | Default | Description | +|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the worker timeout that can be configured for [Puma](https://docs.gitlab.com/omnibus/settings/puma.html#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | +| Fast | 10 seconds | Timeout for fast Gitaly operations used within requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | +| Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly within requests) but preferably not used multiple times within a request. For example, loading blobs. Timeout that should be set between Default and Fast. | diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md index d7c96c295f6..d2f99a51ec3 100644 --- a/doc/user/admin_area/settings/help_page.md +++ b/doc/user/admin_area/settings/help_page.md @@ -5,36 +5,58 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Customizing the 'Help' and login page messages +# Customize the Help and sign-in page messages In large organizations, it is useful to have information about who to contact or where -to go for help. You can customize and display this information on the GitLab server's -`/help` page and on the GitLab login page. +to go for help. You can customize and display this information on the GitLab `/help` page and on +the GitLab sign-in page. -## Adding a help message to the help page +## Add a help message to the Help page -You can add a help message, which is shown on the GitLab `/help` page (e.g., -<https://gitlab.com/help>) in a new section at the top of the `/help` page: +You can add a help message, which is shown at the top of the GitLab `/help` page (for example, +<https://gitlab.com/help>): 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. In the left sidebar, select **Settings > Preferences**, then expand **Help page**. -1. Under **Help page text**, fill in the information you wish to display on `/help`. -1. Save your changes. You can now see the message on `/help`. +1. Under **Additional text to show on the Help page**, fill in the information you wish to display on `/help`. +1. Select **Save changes**. You can now see the message on `/help`. -## Adding a help message to the login page **(STARTER)** +NOTE: +By default, `/help` is visible to unauthenticated users. However, if the +[**Public** visibility level](visibility_and_access_controls.md#restricted-visibility-levels) +is restricted, `/help` is visible only to signed-in users. -You can add a help message, which is shown on the GitLab login page in a new section -titled `Need Help?`, located below the login page message: +## Add a help message to the sign-in page **(STARTER)** + +You can add a help message, which is shown on the GitLab sign-in page. The message appears in a new +section titled **Need Help?**, located below the sign-in page message: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. In the left sidebar, select **Settings > Preferences**, then expand **Help page**. -1. Under **Help text**, fill in the information you wish to display on the login page. +1. Under **Additional text to show on the sign-in page**, fill in the information you wish to + display on the sign-in page. +1. Select **Save changes**. You can now see the message on the sign-in page. + +## Hide marketing-related entries from the Help page + +GitLab marketing-related entries are occasionally shown on the Help page. To hide these entries: -  +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. In the left sidebar, select **Settings > Preferences**, then expand **Help page**. +1. Select the **Hide marketing-related entries from the Help page** checkbox. +1. Select **Save changes**. -1. Save your changes. +## Set a custom Support page URL - +You can specify a custom URL to which users are directed when they: + +- Select **Support** from the Help dropdown. +- Select **See our website for help** on the Help page. + +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. In the left sidebar, select **Settings > Preferences**, then expand **Help page**. +1. Enter the URL in the **Support page URL** field. +1. Select **Save changes**. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/img/enforce_terms.png b/doc/user/admin_area/settings/img/enforce_terms.png Binary files differindex 699e0e63ceb..de1a82275ab 100644 --- a/doc/user/admin_area/settings/img/enforce_terms.png +++ b/doc/user/admin_area/settings/img/enforce_terms.png diff --git a/doc/user/admin_area/settings/img/help_page_help_text_ex_v12_3.png b/doc/user/admin_area/settings/img/help_page_help_text_ex_v12_3.png Binary files differdeleted file mode 100644 index 973be2e8b6e..00000000000 --- a/doc/user/admin_area/settings/img/help_page_help_text_ex_v12_3.png +++ /dev/null diff --git a/doc/user/admin_area/settings/img/help_page_help_text_v12_3.png b/doc/user/admin_area/settings/img/help_page_help_text_v12_3.png Binary files differdeleted file mode 100644 index 8848ea55cf3..00000000000 --- a/doc/user/admin_area/settings/img/help_page_help_text_v12_3.png +++ /dev/null diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 6a5af09358d..d21b6c36224 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -68,7 +68,7 @@ To access the default page for Admin Area settings: | Option | Description | | ------ | ----------- | | [Continuous Integration and Deployment](continuous_integration.md) | Auto DevOps, runners and job artifacts. | -| [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration) **(PREMIUM SELF)** | Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/README.md). This pipeline configuration is run after the project's own configuration. | +| [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration) **(PREMIUM SELF)** | Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/index.md). This pipeline configuration is run after the project's own configuration. | | [Package Registry](continuous_integration.md#package-registry-configuration) | Settings related to the use and experience of using the GitLab Package Registry. Note there are [risks involved](../../packages/container_registry/index.md#use-with-external-container-registries) in enabling some of these settings. | ## Reporting @@ -86,7 +86,7 @@ To access the default page for Admin Area settings: | [Metrics - Grafana](../../../administration/monitoring/performance/grafana_configuration.md#integration-with-gitlab-ui) | Enable and configure Grafana. | | [Profiling - Performance bar](../../../administration/monitoring/performance/performance_bar.md#enable-the-performance-bar-via-the-admin-area) | Enable access to the Performance Bar for a given group. | | [Self monitoring](../../../administration/monitoring/gitlab_self_monitoring_project/index.md#creating-the-self-monitoring-project) | Enable or disable instance self monitoring. | -| [Usage statistics](usage_statistics.md) | Enable or disable version check and usage ping. | +| [Usage statistics](usage_statistics.md) | Enable or disable version check and Service Ping. | | [Pseudonymizer data collection](../../../administration/pseudonymizer.md) **(ULTIMATE)** | Enable or disable the Pseudonymizer data collection. | ## Network diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index ecd259a345c..333e9465c31 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -21,7 +21,7 @@ To access sign-in restriction settings: You can restrict the password authentication for web interface and Git over HTTP(S): -- **Web interface**: When this feature is disabled, the **Standard** sign-in tab is removed and an [external authentication provider](../../../administration/auth/README.md) must be used. +- **Web interface**: When this feature is disabled, the **Standard** sign-in tab is removed and an [external authentication provider](../../../administration/auth/index.md) must be used. - **Git over HTTP(S)**: When this feature is disabled, a [Personal Access Token](../../profile/personal_access_tokens.md) must be used to authenticate. ## Admin Mode diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index e7fa8b1dc40..6f7cb081315 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Third party offers **(FREE SELF)** +# Third-party offers **(FREE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20379) in GitLab Free 11.1. @@ -16,8 +16,9 @@ for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/ To toggle the display of third-party offers: 1. On the top bar, select **Menu >** **{admin}** **Admin**. -1. In the left sidebar, select **Settings**, and expand **Third party offers**. -1. Select **Do not display offers from third parties within GitLab**. +1. On the left sidebar, select **Settings**, and expand **Third-party offers**. +1. Select **Do not display offers from third parties**. +1. Select **Save changes**. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index b5a7ce318ff..c12b720edd2 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -14,7 +14,7 @@ All statistics are opt-out. To enable or disable them: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. In the left sidebar, select **Settings > Metrics and profiling**, and expand **Usage statistics**. -1. Enable or disable **Version check** and **Usage ping**. +1. Enable or disable **Version check** and **Service ping**. 1. Select **Save changes**. ## Network configuration @@ -67,14 +67,14 @@ sequenceDiagram Version Application->>GitLab instance: Response (PNG/SVG) ``` -## Usage Ping **(FREE SELF)** +## Service Ping **(FREE SELF)** -See [Usage Ping guide](../../../development/usage_ping/index.md). +See [Service Ping guide](../../../development/service_ping/index.md). ## Instance-level analytics availability -After usage ping is enabled, GitLab gathers data from other instances and -enables certain [instance-level analytics features](../analytics/index.md) that are dependent on usage ping. +After Service Ping is enabled, GitLab gathers data from other instances and +enables certain [instance-level analytics features](../analytics/index.md) that are dependent on Service Ping. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index 752856371bf..4af33c133a4 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -97,7 +97,8 @@ To change this period: Alternatively, projects that are marked for removal can be deleted immediately. To do so: 1. [Restore the project](../../project/settings/#restore-a-project). -1. Delete the project as described in the [Administering Projects page](../../admin_area/#administering-projects). +1. Delete the project as described in the + [Administering Projects page](../../admin_area/#administering-projects). ## Default project visibility @@ -106,7 +107,8 @@ To set the default visibility levels for new projects: 1. Select the desired default project visibility. 1. Click **Save changes**. -For more details on project visibility, see [Public access](../../../public_access/public_access.md). +For more details on project visibility, see +[Project visibility](../../../public_access/public_access.md). ## Default snippet visibility @@ -115,7 +117,8 @@ To set the default visibility levels for new snippets: 1. Select the desired default snippet visibility. 1. Click **Save changes**. -For more details on snippet visibility, see [Public access](../../../public_access/public_access.md). +For more details on snippet visibility, see +[Project visibility](../../../public_access/public_access.md). ## Default group visibility @@ -124,7 +127,8 @@ To set the default visibility levels for new groups: 1. Select the desired default group visibility. 1. Click **Save changes**. -For more details on group visibility, see [Public access](../../../public_access/public_access.md). +For more details on group visibility, see +[Group visibility](../../group/index.md#group-visibility). ## Restricted visibility levels @@ -133,7 +137,8 @@ To set the restricted visibility levels for projects, snippets, and selected pag 1. Select the desired visibility levels to restrict. 1. Select **Save changes**. -For more details on project visibility, see [Public access](../../../public_access/public_access.md). +For more details on project visibility, see +[Project visibility](../../../public_access/public_access.md). ## Import sources diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index 1fce741cbef..3c80e1f5f2a 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -50,9 +50,9 @@ The following table shows the supported metrics, at which level they are support | Metric | Level | API version | Chart (UI) version | Comments | |---------------------------|---------------------|--------------------------------------|---------------------------------------|-----------| | `deployment_frequency` | Project-level | [13.7+](../../api/dora/metrics.md) | [13.8+](#deployment-frequency-charts) | The [old API endpoint](../../api/dora4_project_analytics.md) was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/323713) in 13.10. | -| `deployment_frequency` | Group-level | [13.10+](../../api/dora/metrics.md) | To be supported | | +| `deployment_frequency` | Group-level | [13.10+](../../api/dora/metrics.md) | [13.12+](#deployment-frequency-charts) | | | `lead_time_for_changes` | Project-level | [13.10+](../../api/dora/metrics.md) | [13.11+](#lead-time-charts) | Unit in seconds. Aggregation method is median. | -| `lead_time_for_changes` | Group-level | [13.10+](../../api/dora/metrics.md) | To be supported | Unit in seconds. Aggregation method is median. | +| `lead_time_for_changes` | Group-level | [13.10+](../../api/dora/metrics.md) | [14.0+](#lead-time-charts) | Unit in seconds. Aggregation method is median. | | `change_failure_rate` | Project/Group-level | To be supported | To be supported | | | `time_to_restore_service` | Project/Group-level | To be supported | To be supported | | diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index c3b3fcba52e..4ad3a03a5b0 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -58,9 +58,9 @@ The **Time** metrics near the top of the page are measured as follows: ## How the stages are measured -Value Stream Analytics uses start events and stop events to measure the time that an issue or merge request spends in each stage. +Value Stream Analytics uses start events and end events to measure the time that an issue or merge request spends in each stage. For example, a stage might start when one label is added to an issue and end when another label is added. -Items aren't included in the stage time calculation if they have not reached the stop event. +Items aren't included in the stage time calculation if they have not reached the end event. | Stage | Description | |---------|---------------| @@ -112,7 +112,7 @@ environments is configured. 1. Push branch, and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) in its description at 14:00 (stop of **Code** stage and start of **Test** and **Review** stages). -1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md) and +1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/index.md) and takes 5 minutes (stop of **Test** stage). 1. Review merge request, ensure that everything is okay, and then merge the merge request at 19:00 (stop of **Review** stage and start of **Staging** stage). diff --git a/doc/user/application_security/api_fuzzing/create_har_files.md b/doc/user/application_security/api_fuzzing/create_har_files.md index 1162984a02d..7940e072420 100644 --- a/doc/user/application_security/api_fuzzing/create_har_files.md +++ b/doc/user/application_security/api_fuzzing/create_har_files.md @@ -12,7 +12,7 @@ requests and HTTP responses. A HAR file's content is JSON formatted, containing with a web site. The file extension `.har` is commonly used. The HAR files can be used to perform [web API Fuzz Testing](index.md#http-archive-har) as part of -your [GitLab CI/CD](../../../ci/README.md) pipelines. +your [GitLab CI/CD](../../../ci/index.md) pipelines. WARNING: A HAR file stores information exchanged between web client and web server. It could also diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 2b2ac76a7af..e35415003c7 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -13,7 +13,7 @@ backend. This helps you discover bugs and potential security issues that other Q miss. We recommend that you use fuzz testing in addition to [GitLab Secure](../index.md)'s -other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run fuzz tests as part your CI/CD workflow. ## When Web API fuzzing runs @@ -134,7 +134,7 @@ To configure API fuzzing in GitLab with an OpenAPI Specification: 1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. [Include](../../../ci/yaml/README.md#includetemplate) +1. [Include](../../../ci/yaml/index.md#includetemplate) the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) in your `.gitlab-ci.yml` file. @@ -200,7 +200,7 @@ To configure API fuzzing to use a HAR file: 1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. [Include](../../../ci/yaml/README.md#includetemplate) +1. [Include](../../../ci/yaml/index.md#includetemplate) the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) in your `.gitlab-ci.yml` file. @@ -271,7 +271,7 @@ To configure API fuzzing to use a Postman Collection file: 1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. [Include](../../../ci/yaml/README.md#includetemplate) +1. [Include](../../../ci/yaml/index.md#includetemplate) the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) in your `.gitlab-ci.yml` file. @@ -400,7 +400,7 @@ To use HTTP basic authentication, two CI/CD variables are added to your `.gitlab - `FUZZAPI_HTTP_USERNAME`: The username for authentication. - `FUZZAPI_HTTP_PASSWORD`: The password for authentication. -For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables) +For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) (for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable as the value for `FUZZAPI_HTTP_PASSWORD`: @@ -438,7 +438,7 @@ outgoing HTTP requests. Follow these steps to provide the bearer token with `FUZZAPI_OVERRIDES_ENV`: -1. [Create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables), +1. [Create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables), for example `TEST_API_BEARERAUTH`, with the value `{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the @@ -780,7 +780,7 @@ variables: ``` In this example `.gitlab-ci.yml`, the `SECRET_OVERRIDES` variable provides the JSON. This is a -[group or instance level CI/CD variable defined in the UI](../../../ci/variables/README.md#add-a-cicd-variable-to-an-instance): +[group or instance level CI/CD variable defined in the UI](../../../ci/variables/index.md#add-a-cicd-variable-to-an-instance): ```yaml stages: diff --git a/doc/user/application_security/cluster_image_scanning/index.md b/doc/user/application_security/cluster_image_scanning/index.md new file mode 100644 index 00000000000..abbe00a85ab --- /dev/null +++ b/doc/user/application_security/cluster_image_scanning/index.md @@ -0,0 +1,281 @@ +--- +type: reference, howto +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Cluster Image Scanning **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 14.1. + +WARNING: +This analyzer is in [Alpha](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha) +and is unstable. The JSON report and CI/CD configuration may be subject to change or breakage +across GitLab releases. + +Your Kubernetes cluster may run workloads based on images that the Container Security analyzer +didn't scan. These images may therefore contain known vulnerabilities. By including an extra job in +your pipeline that scans for those security risks and displays them in the vulnerability report, you +can use GitLab to audit your Kubernetes workloads and environments. + +GitLab provides integration with open-source tools for vulnerability analysis in Kubernetes clusters: + +- [Starboard](https://github.com/aquasecurity/starboard) + +To integrate GitLab with security scanners other than those listed here, see +[Security scanner integration](../../../development/integrations/secure.md). + +You can enable cluster image scanning by [including the CI job](#configuration) +in your existing `.gitlab-ci.yml` file. + +## Prerequisites + +To enable cluster image scanning in your pipeline, you need the following: + +- [GitLab Runner](https://docs.gitlab.com/runner/) + with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) + or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) + executor. +- Docker `18.09.03` or later installed on the same computer as the runner. If you're using the + shared runners on GitLab.com, then this is already the case. +- [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/installation/kubectl/) + installed and configured in your cluster. +- The configuration for accessing your Kubernetes cluster stored in the `CIS_KUBECONFIG` + [configuration variable](#cicd-variables-for-cluster-image-scanning) + with the type set to `File` (see [Configuring the cluster](#configuring-the-cluster)). + +## Configuring the cluster + +1. Create a new service account. + + To properly fetch vulnerabilities from the cluster and to limit analyzer access to the workload, + you must create a new service account with the cluster role limited to `get`, `list`, and `watch` + `vulnerabilityreports` in the Kubernetes cluster: + + ```shell + kubectl apply -f https://gitlab.com/gitlab-org/security-products/analyzers/cluster-image-scanning/-/raw/main/gitlab-vulnerability-viewer-service-account.yaml + ``` + +1. Obtain the Kubernetes API URL. + + Get the API URL by running this command: + + ```shell + API_URL=$(kubectl cluster-info | grep -E 'Kubernetes master|Kubernetes control plane' | awk '/http/ {print $NF}') + ``` + +1. Obtain the CA certificate: + + 1. List the secrets with `kubectl get secrets`. One should have a name similar to + `default-token-xxxxx`. Copy that token name for use below. + + 1. Run this command to get the certificate: + + ```shell + CA_CERTIFICATE=$(kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}") + ``` + +1. Obtain the service account token: + + ```shell + TOKEN=$(kubectl -n kube-system get secret $(kubectl -n kube-system get secret | grep gitlab-vulnerability-viewer | awk '{print $1}') -o jsonpath="{.data.token}" | base64 --decode) + ``` + +1. Generate the value for the `CIS_KUBECONFIG` variable. Copy the printed value from the output: + + ```shell + echo " + --- + apiVersion: v1 + kind: Config + clusters: + - name: gitlab-vulnerabilities-viewer + cluster: + server: $API_URL + certificate-authority-data: $CA_CERTIFICATE + contexts: + - name: gitlab-vulnerabilities-viewer + context: + cluster: gitlab-vulnerabilities-viewer + namespace: default + user: gitlab-vulnerabilities-viewer + current-context: gitlab-vulnerabilities-viewer + users: + - name: gitlab-vulnerabilities-viewer + user: + token: $TOKEN + " + ``` + +1. Set the CI/CD variable: + + 1. Navigate to your project's **Settings > CI/CD**. + + 1. Expand the **Variables** section. + + 1. Select **Add variable** and fill in the details: + + - **Key**: `CIS_KUBECONFIG`. + - **Value**: `generated value` + - **Type**: `File` + +WARNING: +The `CIS_KUBECONFIG` variable is accessible by all jobs executed for your project. Mark the +`Protect variable` flag to export this variable to pipelines running on protected branches and tags +only. You can apply additional protection to your cluster by +[restricting service account access to a single namespace](https://kubernetes.io/docs/reference/access-authn-authz/rbac/), +and [configuring Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/configuration/#install-modes) +to install in restricted mode. + +## Configuration + +To include the `Cluster-Image-Scanning.gitlab-ci.yml` template (GitLab 14.1 and later), add the +following to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: Security/Cluster-Image-Scanning.gitlab-ci.yml +``` + +The included template: + +- Creates a `cluster_image_scanning` job in your CI/CD pipeline. +- Connects to your Kubernetes cluster with credentials provided in the `CIS_KUBECONFIG` variable and + fetches vulnerabilities found by [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/). + +GitLab saves the results as a +[Cluster Image Scanning report artifact](../../../ci/yaml/index.md#artifactsreportscluster_image_scanning) +that you can download and analyze later. When downloading, you always receive the most recent +artifact. + +### Customize the cluster image scanning settings + +You can customize how GitLab scans your cluster. For example, to restrict the analyzer to get +results for only a certain workload, use the [`variables`](../../../ci/yaml/index.md#variables) +parameter in your `.gitlab-ci.yml` to set [CI/CD variables](#cicd-variables-for-cluster-image-scanning). +The variables you set in your `.gitlab-ci.yml` overwrite those in +`Cluster-Image-Scanning.gitlab-ci.yml`. + +#### CI/CD variables for cluster image scanning + +You can [configure](#customize-the-cluster-image-scanning-settings) analyzers by using the following CI/CD variables: + +| CI/CD Variable | Default | Description | +| ------------------------------ | ------------- | ----------- | +| `CIS_KUBECONFIG` | `""` | File used to configure access to the Kubernetes cluster. See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) for more details. | +| `CIS_CONTAINER_NAME` | `""` | Name of the container used in the Kubernetes resource you want to filter vulnerabilities for. For example, `alpine`. | +| `CIS_RESOURCE_NAME` | `""` | Name of the Kubernetes resource you want to filter vulnerabilities for. For example, `nginx`. | +| `CIS_RESOURCE_NAMESPACE` | `""` | Namespace of the Kubernetes resource you want to filter vulnerabilities for. For example, `production`. | +| `CIS_RESOURCE_KIND` | `""` | Kind of the Kubernetes resource you want to filter vulnerabilities for. For example, `deployment`. | + +### Override the cluster image scanning template + +If you want to override the job definition (for example, to change properties like `variables`), you +must declare and override a job after the template inclusion, and then +specify any additional keys. + +This example sets `CIS_RESOURCE_NAME` to `nginx`: + +```yaml +include: + - template: Security/Cluster-Image-Scanning.gitlab-ci.yml + +cluster_image_scanning: + variables: + CIS_RESOURCE_NAME: nginx +``` + +### Connect with Kubernetes cluster associated to the project + +If you want to connect to the Kubernetes cluster associated with the project and run Cluster Image Scanning jobs without +configuring the `CIS_KUBECONFIG` variable, you must extend `cluster_image_scanning` and specify the environment you want to scan. + +This example configures the `cluster_image_scanning` job to scan the Kubernetes cluster connected with the `staging` environment: + +```yaml +cluster_image_scanning: + environment: + name: staging + action: prepare +``` + +## Reports JSON format + +The cluster image scanning tool emits a JSON report file. For more information, see the +[schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/container-scanning-report-format.json). + +Here's an example cluster image scanning report: + +```json-doc +{{ + "version": "14.0.2", + "scan": { + "scanner": { + "id": "starboard_trivy", + "name": "Trivy (using Starboard Operator)", + "url": "https://github.com/aquasecurity/starboard", + "vendor": { + "name": "GitLab" + }, + "version": "0.16.0" + }, + "start_time": "2021-04-28T12:47:00Z", + "end_time": "2021-04-28T12:47:00Z", + "type": "cluster_image_scanning", + "status": "success" + }, + "vulnerabilities": [ + { + "id": "c15f22205ee842184c2d55f1a207b3708283353f85083d66c34379c709b0ac9d", + "category": "cluster_image_scanning", + "message": "CVE-2011-3374 in apt", + "description": "", + "cve": "library/nginx:1.18:apt:CVE-2011-3374", + "severity": "Low", + "confidence": "Unknown", + "solution": "Upgrade apt from 1.8.2.2", + "scanner": { + "id": "starboard_trivy", + "name": "Trivy (using Starboard Operator)" + }, + "location": { + "dependency": { + "package": { + "name": "apt" + }, + "version": "1.8.2.2" + }, + "operating_system": "library/nginx:1.18", + "image": "index.docker.io/library/nginx:1.18" + }, + "identifiers": [ + { + "type": "cve", + "name": "CVE-2011-3374", + "value": "CVE-2011-3374", + "url": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-3374" + } + ], + "links": [ + "https://avd.aquasec.com/nvd/cve-2011-3374" + ] + } + ] +} +``` + +## Security Dashboard + +The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all +the security vulnerabilities in your groups, projects, and pipelines. + +## Interacting with the vulnerabilities + +After a vulnerability is found, you can [address it](../vulnerabilities/index.md). + +## Troubleshooting + +### Getting warning message `gl-cluster-image-scanning-report.json: no matching files` + +For information on this error, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 8c34303ca00..3cc88a40b6f 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -25,6 +25,32 @@ For each security control the page displays: - **Security Control:** Name, description, and a documentation link. - **Manage:** A management option or a documentation link. +## UI redesign + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326926) in 14.0 for GitLab Free and Premium, behind a feature flag, disabled by default. +> - Enabled on GitLab.com for Free & Premium. +> - Recommended for production use. +> - It can be enabled or disabled for a single project. +> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-ui-redesign). **(FREE SELF)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333109) in 14.1 for GitLab Ultimate, behind a feature flag, disabled by default. +> - Disabled on GitLab.com. +> - Not recommended for production use. +> - It can be enabled or disabled for a single project. +> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-ui-redesign-for-ultimate). **(ULTIMATE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +The Security Configuration page has been redesigned in GitLab Free and Premium. +The same functionality exists as before, but presented in a more extensible +way. + +For each security control the page displays: + +- Its name, description and a documentation link. +- Whether or not it is available. +- A configuration button or a link to its configuration guide. + ## Status **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. @@ -52,3 +78,56 @@ You can configure the following security controls: - Secret Detection - Select **Configure via Merge Request** to create a merge request with the changes required to enable Secret Detection. For more details, see [Enable Secret Detection via an automatic merge request](../secret_detection/index.md#enable-secret-detection-via-an-automatic-merge-request). +- Dependency Scanning + - Select **Configure via Merge Request** to create a merge request with the changes required to + enable Dependency Scanning. For more details, see [Enable Dependency Scanning via an automatic merge request](../dependency_scanning/index.md#enable-dependency-scanning-via-an-automatic-merge-request). + +## Enable or disable UI redesign **(FREE SELF)** + +The Security Configuration redesign is under development, but is ready for +production use. It is deployed behind a feature flag that is **disabled by +default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can enable it. + +To enable it: + +```ruby +# For the instance +Feature.enable(:security_configuration_redesign) +# For a single project +Feature.enable(:security_configuration_redesign, Project.find(<project id>)) +``` + +To disable it: + +```ruby +# For the instance +Feature.disable(:security_configuration_redesign) +# For a single project +Feature.disable(:security_configuration_redesign, Project.find(<project id>)) +``` + +## Enable or disable UI redesign for Ultimate **(ULTIMATE SELF)** + +The Security Configuration redesign is under development, and is not ready for +production use. It is deployed behind a feature flag that is **disabled by +default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can enable it. + +To enable it: + +```ruby +# For the instance +Feature.enable(:security_configuration_redesign_ee) +# For a single project +Feature.enable(:security_configuration_redesign_ee, Project.find(<project id>)) +``` + +To disable it: + +```ruby +# For the instance +Feature.disable(:security_configuration_redesign_ee) +# For a single project +Feature.disable(:security_configuration_redesign_ee, Project.find(<project id>)) +``` diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 323a064c3e4..90e1e4b025c 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -59,7 +59,7 @@ To enable container scanning in your pipeline, you need the following: How you enable container scanning depends on your GitLab version: -- GitLab 11.9 and later: [Include](../../../ci/yaml/README.md#includetemplate) the +- GitLab 11.9 and later: [Include](../../../ci/yaml/index.md#includetemplate) the [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) that comes with your GitLab installation. - GitLab versions earlier than 11.9: Copy and use the job from the @@ -73,8 +73,8 @@ Other changes: [`centos:centos8`](https://hub.docker.com/_/centos) as the new base. It also removes the use of the [start.sh](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/merge_requests/77) script and instead executes the analyzer by default. Any customizations made to the - `container_scanning` job's [`before_script`](../../../ci/yaml/README.md#before_script) - and [`after_script`](../../../ci/yaml/README.md#after_script) + `container_scanning` job's [`before_script`](../../../ci/yaml/index.md#before_script) + and [`after_script`](../../../ci/yaml/index.md#after_script) blocks may not work with the new version. To roll back to the previous [`alpine:3.11.3`](https://hub.docker.com/_/alpine)-based Docker image, you can specify the major version through the [`CS_MAJOR_VERSION`](#available-cicd-variables) variable. @@ -101,7 +101,7 @@ The included template: (see [requirements](#requirements)) and scans it for possible vulnerabilities. GitLab saves the results as a -[Container Scanning report artifact](../../../ci/yaml/README.md#artifactsreportscontainer_scanning) +[Container Scanning report artifact](../../../ci/yaml/index.md#artifactsreportscontainer_scanning) that you can download and analyze later. When downloading, you always receive the most-recent artifact. @@ -130,12 +130,12 @@ include: There may be cases where you want to customize how GitLab scans your containers. For example, you may want to enable more verbose output, access a Docker registry that requires -authentication, and more. To change such settings, use the [`variables`](../../../ci/yaml/README.md#variables) +authentication, and more. To change such settings, use the [`variables`](../../../ci/yaml/index.md#variables) parameter in your `.gitlab-ci.yml` to set [CI/CD variables](#available-cicd-variables). The variables you set in your `.gitlab-ci.yml` overwrite those in `Container-Scanning.gitlab-ci.yml`. -This example [includes](../../../ci/yaml/README.md#include) the container scanning template and +This example [includes](../../../ci/yaml/index.md#include) the container scanning template and enables verbose output for the analyzer: ```yaml @@ -172,6 +172,21 @@ Support depends on the scanner: - [Grype](https://github.com/anchore/grype#grype) - [Trivy](https://aquasecurity.github.io/trivy/latest/vuln-detection/os/) (Default). +#### UBI-based images + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5775) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 14.1. + +GitLab also offers [Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) +versions of the container-scanning images. You can therefore replace standard images with UBI-based +images. To configure the images, set the `CS_ANALYZER_IMAGE` variable to the standard tag plus the +`-ubi` extension. + +| Scanner name | `CS_ANALYZER_IMAGE` | +| --------------- | ------------------- | +| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:4-ubi` | +| Grype | `registry.gitlab.com/security-products/container-scanning/grype:4-ubi` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4-ubi` | + ### Overriding the container scanning template If you want to override the job definition (for example, to change properties like `variables`), you @@ -189,11 +204,6 @@ container_scanning: GIT_STRATEGY: fetch ``` -WARNING: -GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#only--except). -When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) -instead. - ### Change scanners The container-scanning analyzer can use different scanners, depending on the value of the @@ -256,7 +266,7 @@ container_scanning: -----END CERTIFICATE----- ``` -The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/README.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. +The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. ### Vulnerability allowlisting diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index b46547b6828..679d20a6394 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -13,7 +13,7 @@ random inputs to an instrumented version of your application in an effort to cau behavior, such as a crash. Such behavior indicates a bug that you should address. We recommend that you use fuzz testing in addition to the other security scanners in [GitLab Secure](../index.md) -and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run your coverage-guided fuzz tests as part your CI/CD workflow. You can take advantage of coverage-guided fuzzing by including the CI job in your existing `.gitlab-ci.yml` file. @@ -38,7 +38,7 @@ Docker image with the fuzz engine to run your app. ## Configuration To enable fuzzing, you must -[include](../../../ci/yaml/README.md#includetemplate) +[include](../../../ci/yaml/index.md#includetemplate) the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml) provided as part of your GitLab installation. @@ -59,8 +59,8 @@ my_fuzz_target: - ./gitlab-cov-fuzz run --regression=$REGRESSION -- <your fuzz target> ``` -The included template makes available the [hidden job](../../../ci/yaml/README.md#hide-jobs) -`.fuzz_base`, which you must [extend](../../../ci/yaml/README.md#extends) for each of your fuzz +The included template makes available the [hidden job](../../../ci/yaml/index.md#hide-jobs) +`.fuzz_base`, which you must [extend](../../../ci/yaml/index.md#extends) for each of your fuzz targets. Each fuzz target **must** have a separate job. For example, the [go-fuzzing-example project](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example) contains one job that extends `.fuzz_base` for its single fuzz target. @@ -192,7 +192,7 @@ To use coverage fuzzing in an offline environment, follow these steps: ### Continuous fuzzing (long-running asynchronous fuzzing jobs) It's also possible to run the fuzzing jobs longer and without blocking your main pipeline. This -configuration uses the GitLab [parent-child pipelines](../../../ci/parent_child_pipelines.md). +configuration uses the GitLab [parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md). The full example is available in the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing#running-go-fuzz-from-ci). This example uses Go, but is applicable for any other supported languages. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index 072f6fba4ba..1288db21880 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -21,7 +21,7 @@ Crawling continues by taking more snapshots and finding subsequent actions. The benefit of crawling by following user actions in a browser is that the crawler can interact with the target application much like a real user would, identifying complex flows that traditional web crawlers don’t understand. This results in better coverage of the website. -Scanning a web application with both the browser-based crawler and GitLab DAST should provide greater coverage, compared with only GitLab DAST. The new crawler does not support API scanning or the DAST AJAX crawler. +Using the browser-based crawler should provide greater coverage for most web applications, compared with the current DAST AJAX crawler. The new crawler replaces the AJAX crawler and is specifically designed to maximize crawl coverage in modern web applications. While both crawlers are currently used in conjunction with the existing DAST scanner, the combination of the browser-based crawler with the current DAST scanner is much more effective at finding and testing every page in an application. ## Enable browser-based crawler @@ -63,8 +63,8 @@ The browser-based crawler can be configured using CI/CD variables. The [DAST variables](index.md#available-cicd-variables) `SECURE_ANALYZERS_PREFIX`, `DAST_FULL_SCAN_ENABLED`, `DAST_AUTO_UPDATE_ADDONS`, `DAST_EXCLUDE_RULES`, `DAST_REQUEST_HEADERS`, `DAST_HTML_REPORT`, `DAST_MARKDOWN_REPORT`, `DAST_XML_REPORT`, `DAST_AUTH_URL`, `DAST_USERNAME`, `DAST_PASSWORD`, `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, `DAST_FIRST_SUBMIT_FIELD`, `DAST_SUBMIT_FIELD`, `DAST_EXCLUDE_URLS`, `DAST_AUTH_VERIFICATION_URL`, `DAST_BROWSER_AUTH_VERIFICATION_SELECTOR`, `DAST_BROWSER_AUTH_VERIFICATION_LOGIN_FORM`, `DAST_BROWSER_AUTH_REPORT`, -`DAST_INCLUDE_ALPHA_VULNERABILITIES`, `DAST_PATHS_FILE`, `DAST_PATHS`, `DAST_ZAP_CLI_OPTIONS`, and `DAST_ZAP_LOG_CONFIGURATION` are also compatible with browser-based crawler scans. - +`DAST_INCLUDE_ALPHA_VULNERABILITIES`, `DAST_PATHS_FILE`, `DAST_PATHS`, `DAST_ZAP_CLI_OPTIONS`, and `DAST_ZAP_LOG_CONFIGURATION` are also compatible with browser-based crawler scans. + ## Vulnerability detection While the browser-based crawler crawls modern web applications efficiently, vulnerability detection is still managed by the standard DAST/Zed Attack Proxy (ZAP) solution. diff --git a/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png b/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png Binary files differindex 132c9f9c991..3369956a5ed 100644 --- a/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png +++ b/doc/user/application_security/dast/img/dast_auth_browser_scan_highlight.png diff --git a/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png b/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png Binary files differindex 4e1dca626bc..34e7a2e4ab4 100644 --- a/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png +++ b/doc/user/application_security/dast/img/dast_auth_browser_scan_search_elements.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 675d01373d4..4b10f03fec3 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -50,7 +50,7 @@ results. On failure, the analyzer outputs an ## Prerequisites -- [GitLab Runner](../../../ci/runners/README.md) available, with the +- [GitLab Runner](../../../ci/runners/index.md) available, with the [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). - Target application deployed. For more details, read [Deployment options](#deployment-options). @@ -145,6 +145,7 @@ To enable DAST to run automatically, either: by [Auto DevOps](../../../topics/autodevops/index.md)). - [Include the DAST template](#include-the-dast-template) in your existing `.gitlab-ci.yml` file. +- [Configure DAST using the UI](#configure-dast-using-the-ui). ### DAST job order @@ -202,7 +203,7 @@ To include the DAST template: 1. Add the template to GitLab, based on your version of GitLab: - - In GitLab 11.9 and later, [include](../../../ci/yaml/README.md#includetemplate) + - In GitLab 11.9 and later, [include](../../../ci/yaml/index.md#includetemplate) the template by adding the following to your `.gitlab-ci.yml` file: ```yaml @@ -218,7 +219,7 @@ To include the DAST template: 1. Define the URL to be scanned by DAST by using one of these methods: - - Set the `DAST_WEBSITE` [CI/CD variable](../../../ci/yaml/README.md#variables). + - Set the `DAST_WEBSITE` [CI/CD variable](../../../ci/yaml/index.md#variables). If set, this value takes precedence. - Add the URL in an `environment_url.txt` file at the root of your project. This is @@ -247,10 +248,10 @@ The included template creates a `dast` job in your CI/CD pipeline and scans your project's running application for possible vulnerabilities. The results are saved as a -[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast) +[DAST report artifact](../../../ci/yaml/index.md#artifactsreportsdast) that you can later download and analyze. Due to implementation limitations, we always take the latest DAST artifact available. Behind the scenes, the -[GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast) +[GitLab DAST Docker image](https://gitlab.com/security-products/dast) is used to run the tests on the specified URL and scan it for possible vulnerabilities. @@ -262,9 +263,31 @@ image. Using the `DAST_VERSION` variable, you can choose how DAST updates: - Only update fixes by pinning to a minor version (such as `1.6`). - Prevent all updates by pinning to a specific version (such as `1.6.4`). -Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases) +Find the latest DAST versions on the [Releases](https://gitlab.com/security-products/dast/-/releases) page. +#### Configure DAST using the UI + +You can enable or configure DAST settings using the UI. The generated settings are formatted so they +can be conveniently pasted into the `.gitlab-ci.yml` file. + +1. From the project's home page, go to **Security & Compliance > Configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Enable DAST** or + **Configure DAST**. +1. Select the desired **Scanner profile**, or select **Create scanner profile** and save a + scanner profile. For more details, see [scanner profiles](#scanner-profile). +1. Select the desired **Site profile**, or select **Create site profile** and save a site + profile. For more details, see [site profiles](#site-profile). +1. Select **Generate code snippet**. A modal opens with the YAML snippet corresponding to the + options you selected. +1. Do one of the following: + 1. Select **Copy code only** to copy the snippet to your clipboard. + 1. Select **Copy code and open `.gitlab-ci.yml` file** to copy the snippet to your clipboard. The + CI/CD Editor then opens. + 1. Paste the snippet into the `.gitlab-ci.yml` file. + 1. Select the **Lint** tab to confirm the edited `.gitlab-ci.yml` file is valid. + 1. Select **Commit changes**. + #### Crawling web applications dependent on JavaScript GitLab has released a new browser-based crawler, an add-on to DAST that uses a browser to crawl web applications for content. This crawler replaces the standard DAST Spider and Ajax Crawler, and uses the same authentication mechanisms as a normal DAST scan. @@ -487,11 +510,11 @@ To view details of vulnerabilities detected by DAST: ### Customizing the DAST settings WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. The DAST settings can be changed through CI/CD variables by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. These variables are documented in [available variables](#available-cicd-variables). For example: @@ -505,7 +528,7 @@ variables: DAST_SPIDER_MINS: 120 ``` -Because the template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline +Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline configuration, the last mention of the variable takes precedence. #### Enabling and disabling rules @@ -523,7 +546,7 @@ DAST scan with both configured exits with an error. By default, several rules are disabled because they either take a long time to run or frequently generate false positives. The complete list of disabled rules -can be found in [exclude_rules.yml](https://gitlab.com/gitlab-org/security-products/dast/-/blob/master/src/config/exclude_rules.yml). +can be found in [exclude_rules.yml](https://gitlab.com/gitlab-org/security-products/dast/-/blob/main/src/config/exclude_rules.yml). ### Hide sensitive information @@ -550,7 +573,7 @@ you periodically confirm the scanner's authentication is still working, as this time due to authentication changes to the application. Create masked CI/CD variables to pass the credentials that DAST uses. -To create masked variables for the username and password, see [Create a custom variable in the UI](../../../ci/variables/README.md#custom-cicd-variables). +To create masked variables for the username and password, see [Create a custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables). The key of the username variable must be `DAST_USERNAME`, and the key of the password variable must be `DAST_PASSWORD`. @@ -570,6 +593,7 @@ dast: variables: DAST_WEBSITE: "https://example.com" DAST_AUTH_URL: "https://login.example.com/" + DAST_BROWSER_PATH_TO_LOGIN_FORM: "css:.navigation-menu,css:.login-menu-item" # optional list of selectors that should be clicked on prior to attempting to input username/password into the sign-in HTML form DAST_USERNAME: "admin" DAST_PASSWORD: "P@55w0rd!" DAST_USERNAME_FIELD: "name:username" # a selector describing the element containing the username field at the sign-in HTML form @@ -646,7 +670,7 @@ dast: DAST_WEBSITE: "https://example.com" ... DAST_AUTH_VERIFICATION_URL: "https://example.com/user/welcome" -``` +``` #### Verify based on presence of an element @@ -664,7 +688,7 @@ dast: DAST_WEBSITE: "https://example.com" ... DAST_AUTH_VERIFICATION_SELECTOR: "css:.welcome-user" -``` +``` #### Verify based on presence of a login form @@ -682,7 +706,38 @@ dast: DAST_WEBSITE: "https://example.com" ... DAST_AUTH_VERIFICATION_LOGIN_FORM: "true" -``` +``` + +### View the login form + +Many web applications show the user the login form in a pop-up (modal) window. +For these applications, navigating to the form requires both: + +- A starting URL. +- A list of elements to click to display the modal window. + +When `DAST_BROWSER_PATH_TO_LOGIN_FORM` is present, like in this example: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://my.site.com" + ... + DAST_AUTH_URL: "https://my.site.com/admin" + DAST_BROWSER_PATH_TO_LOGIN_FORM: "css:.navigation-menu,css:.login-menu-item" +``` + +DAST performs these actions: + +1. Load the `DAST_AUTH_URL` page, such as `https://my.site.com/admin`. +1. After the page loads, DAST selects elements found by the selectors described + in `DAST_BROWSER_PATH_TO_LOGIN_FORM`. This example opens the navigation menu + and selects the login menu, to display the login modal window. +1. To continue the authentication process, DAST fills in the username and password + on the login form. ### Configure the authentication debug output @@ -702,55 +757,60 @@ dast: DAST_AUTH_REPORT: "true" artifacts: paths: [gl-dast-debug-auth-report.html] + when: always ``` ### Available CI/CD variables -DAST can be [configured](#customizing-the-dast-settings) using CI/CD variables. - -| CI/CD variable | Type | Description | -|:--------------------------------------------|:--------------|:-----------------------------------| -| `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. | -| `DAST_WEBSITE` (**1**) | URL | The URL of the website to scan. `DAST_API_OPENAPI` must be specified if this is omitted. | -| `DAST_API_OPENAPI` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | -| `DAST_API_SPECIFICATION` (**1**) | URL or string | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290241) in GitLab 13.12 and replaced by `DAST_API_OPENAPI`. To be removed in GitLab 15.0. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | -| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. | -| `DAST_AUTH_URL` (**1**) | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. | -| `DAST_AUTH_VERIFICATION_URL` (**1**) | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST exits if it cannot access the URL. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | -| `DAST_USERNAME` (**1**) | string | The username to enter into the username field on the sign-in HTML form. | -| `DAST_PASSWORD` (**1**) | string | The password to enter into the password field on the sign-in HTML form. | -| `DAST_USERNAME_FIELD` (**1**) | selector | A selector describing the username field on the sign-in HTML form. Example: `id:user` | -| `DAST_PASSWORD_FIELD` (**1**) | selector | A selector describing the password field on the sign-in HTML form. Example: `css:.password-field` | -| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. | -| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | -| `DAST_EXCLUDE_URLS` (**1**) | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | -| `DAST_FULL_SCAN_ENABLED` (**1**) | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | -| `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | -| `DAST_API_HOST_OVERRIDE` (**1**) | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` | -| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | -| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. | -| `DAST_REQUEST_HEADERS` (**1**) | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | -| `DAST_DEBUG` (**1**) | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_TARGET_AVAILABILITY_TIMEOUT` (**1**) | number | Time limit in seconds to wait for target availability. -| `DAST_SPIDER_MINS` (**1**) | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_USE_AJAX_SPIDER` (**1**) | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. | -| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. | -| `DAST_SUBMIT_FIELD` | selector | A selector describing the element that when clicked submits the login form, or the password form of a multi-page login process. Example: `xpath://input[@value='Login']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | -| `DAST_FIRST_SUBMIT_FIELD` | selector | A selector describing the element that when clicked submits the username form of a multi-page login process. Example: `.submit`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | -| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | -| `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` | -| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. | -| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. [Introduced](https://gitlab.com/gitlab-org/security-products/dast/-/merge_requests/433) in GitLab 13.12. | -| `DAST_AUTH_REPORT` | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | -| `DAST_AUTH_VERIFICATION_SELECTOR` | selector | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo` | -| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the lack of a login form once the login form has been submitted. | - -1. DAST CI/CD variable available to an on-demand scan. +You can use CI/CD variables to customize DAST. + +| CI/CD variable | Type | Description | +|:------------------------------------------------|:--------------|:-------------------------------| +| `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. | +| `DAST_WEBSITE` <sup>1</sup> | URL | The URL of the website to scan. `DAST_API_OPENAPI` must be specified if this is omitted. | +| `DAST_API_OPENAPI` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290241) in GitLab 13.12 and replaced by `DAST_API_OPENAPI`. To be removed in GitLab 15.0. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. | +| `DAST_AUTH_URL` <sup>1</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. | +| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1</sup> | selector | Comma-separated list of selectors that will be clicked on prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | +| `DAST_USERNAME` <sup>1</sup> | string | The username to authenticate to in the website. | +| `DAST_PASSWORD` <sup>1</sup> | string | The password to authenticate to in the website. | +| `DAST_USERNAME_FIELD` <sup>1</sup> | string | The name of username field at the sign-in HTML form. | +| `DAST_PASSWORD_FIELD` <sup>1</sup> | string | The name of password field at the sign-in HTML form. | +| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. | +| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | +| `DAST_EXCLUDE_URLS` <sup>1</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | +| `DAST_FULL_SCAN_ENABLED` <sup>1</sup> | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | +| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293595)** in GitLab 14.0. Set to `true` to require domain validation when running DAST full scans. Not supported for API scans. Default: `false` | +| `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | +| `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` | +| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. | +| `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | +| `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_TARGET_AVAILABILITY_TIMEOUT` <sup>1</sup> | number | Time limit in seconds to wait for target availability. | +| `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. | +| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. | +| `DAST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when clicked submits the login form or the password form of a multi-page login process. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | +| `DAST_FIRST_SUBMIT_FIELD` | string | The `id` or `name` of the element that when clicked submits the username form of a multi-page login process. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. | +| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | +| `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` | +| `DAST_AUTH_EXCLUDE_URLS` | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | +| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. | +| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. [Introduced](https://gitlab.com/gitlab-org/security-products/dast/-/merge_requests/433) in GitLab 13.12. | +| `DAST_AUTH_REPORT` | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | +| `DAST_AUTH_VERIFICATION_URL` <sup>1</sup> | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST exits if it cannot access the URL. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | +| `DAST_AUTH_VERIFICATION_SELECTOR` | selector | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo` | +| `DAST_AUTH_VERIFICATION_LOGIN_FORM` | boolean | Verifies successful authentication by checking for the lack of a login form once the login form has been submitted. | +| `DAST_ADVERTISE_SCAN` | boolean | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. | + +1. Available to an on-demand DAST scan. #### Selectors @@ -778,7 +838,8 @@ Chrome DevTools element selector tool is an effective way to find a selector.  1. Once highlighted, you can see the element's details, including attributes that would make a good candidate for a selector. -In this example, the `id="user_login"` appears to be a good candidate. You can use this as a selector as the DAST username field by setting `DAST_USERNAME_FIELD: "css:[id=user_login]"`, or more simply, `DAST_USERNAME_FIELD: "id:user_login"`. +In this example, the `id="user_login"` appears to be a good candidate. You can use this as a selector as the DAST username field by setting +`DAST_USERNAME_FIELD: "id:user_login"`. ##### Choose the right selector @@ -788,9 +849,9 @@ In order of preference, it is recommended to choose as selectors: - `id` fields. These are generally unique on a page, and rarely change. - `name` fields. These are generally unique on a page, and rarely change. -- `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field. +- `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field. - Presence of field specific data attributes, such as the selector, `"css:[data-username]"` when the `data-username` field has any value on the username field. -- Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`. +- Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`. When using selectors to locate specific fields we recommend you avoid searching on: @@ -880,6 +941,8 @@ Debug mode of the ZAP server can be enabled using the `DAST_ZAP_LOG_CONFIGURATIO The following table outlines examples of values that can be set and the effect that they have on the output that is logged. Multiple values can be specified, separated by semicolons. +For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG`. + | Log configuration value | Effect | |-------------------------------------------------- | ----------------------------------------------------------------- | | `log4j.rootLogger=DEBUG` | Enable all debug logging statements. | @@ -901,8 +964,8 @@ To use DAST in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#prerequisites). - Docker Container Registry with a locally available copy of the DAST - [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the - [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). + [container image](https://gitlab.com/security-products/dast), found in the + [DAST container registry](https://gitlab.com/security-products/dast/container_registry). Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local @@ -915,7 +978,7 @@ enables the use of updated scanners in your CI/CD pipelines. For DAST, import the following default DAST analyzer image from `registry.gitlab.com` to your [local Docker container registry](../../packages/container_registry/index.md): -- `registry.gitlab.com/gitlab-org/security-products/dast:latest` +- `registry.gitlab.com/security-products/dast:latest` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved @@ -953,6 +1016,7 @@ Alternatively, you can use the CI/CD variable `SECURE_ANALYZERS_PREFIX` to overr > - The saved scans feature was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5100) in GitLab 13.9. > - The option to select a branch was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4847) in GitLab 13.10. > - DAST branch selection [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322672) in GitLab 13.11. +> - Auditing for DAST profile management was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger the scan. You must start it manually. @@ -1276,6 +1340,13 @@ If a scanner profile is linked to a security policy, a user cannot delete the pr page. See [Scan Policies](../policies/index.md) for more information. +### Auditing + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. + +The creation, updating, and deletion of DAST profiles, DAST scanner profiles, +and DAST site profiles are included in the [audit log](../../../administration/audit_events.md). + ## Reports The DAST tool outputs a report file in JSON format by default. However, this tool can also generate reports in diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index 9a6e1e73330..48a784e0d03 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -7,11 +7,11 @@ type: reference, howto # DAST API **(ULTIMATE)** -You can add dynamic application security testing of web APIs to your [GitLab CI/CD](../../../ci/README.md) pipelines. +You can add dynamic application security testing of web APIs to your [GitLab CI/CD](../../../ci/index.md) pipelines. This helps you discover bugs and potential security issues that other QA processes may miss. We recommend that you use DAST API testing in addition to [GitLab Secure](../index.md)'s -other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), you can run DAST API tests as part your CI/CD workflow. ## Requirements @@ -85,7 +85,7 @@ the body generation is limited to these body types: Follow these steps to configure DAST API in GitLab with an OpenAPI specification: -1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) +1. To use DAST API, you must [include](../../../ci/yaml/index.md#includetemplate) the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) that's provided as part of your GitLab installation. Add the following to your `.gitlab-ci.yml` file: @@ -184,7 +184,7 @@ cookies. We recommend that you review the HAR file contents before adding them t Follow these steps to configure DAST API to use a HAR file that provides information about the target API to test: -1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) +1. To use DAST API, you must [include](../../../ci/yaml/index.md#includetemplate) the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) that's provided as part of your GitLab installation. To do so, add the following to your `.gitlab-ci.yml` file: @@ -284,7 +284,7 @@ them to a repository. Follow these steps to configure DAST API to use a Postman Collection file that provides information about the target API to test: -1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) +1. To use DAST API, you must [include](../../../ci/yaml/index.md#includetemplate) the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) that's provided as part of your GitLab installation. To do so, add the following to your `.gitlab-ci.yml` file: @@ -435,7 +435,7 @@ To use HTTP basic authentication, two CI/CD variables are added to your `.gitlab - `DAST_API_HTTP_USERNAME`: The username for authentication. - `DAST_API_HTTP_PASSWORD`: The password for authentication. -For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables) +For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) (for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable as the value for `DAST_API_HTTP_PASSWORD`: @@ -473,7 +473,7 @@ outgoing HTTP requests. Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: -1. [Create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables), +1. [Create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables), for example `TEST_API_BEARERAUTH`, with the value `{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the @@ -849,7 +849,7 @@ variables: ``` In this example `.gitlab-ci.yml`, the `SECRET_OVERRIDES` variable provides the JSON. This is a -[group or instance level CI/CD variable defined in the UI](../../../ci/variables/README.md#add-a-cicd-variable-to-an-instance): +[group or instance level CI/CD variable defined in the UI](../../../ci/variables/index.md#add-a-cicd-variable-to-an-instance): ```yaml stages: diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index fcefba943ad..9fc90c427c5 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -13,7 +13,7 @@ Use the dependency list to review your project's dependencies and key details about those dependencies, including their known vulnerabilities. It is a collection of dependencies in your project, including existing and new findings. To see the dependency list, go to your project and select **Security & Compliance > Dependency List**. This information is sometimes referred to as a Software Bill of Materials or SBoM / BOM. -The dependency list only shows the results of the last successful pipeline to run on the default branch. This is why we recommend not changing the default behavior of allowing the secure jobs to fail. +The dependency list only shows the results of the last successful pipeline to run on the default branch. This is why we recommend not changing the default behavior of allowing the secure jobs to fail. ## Prerequisites diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 96fc085e7c6..76a14aae715 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -16,7 +16,7 @@ vulnerable. You can then take action to protect your application. ## Overview -If you're using [GitLab CI/CD](../../../ci/README.md), you can use dependency scanning to analyze +If you're using [GitLab CI/CD](../../../ci/index.md), you can use dependency scanning to analyze your dependencies for known vulnerabilities. GitLab scans all dependencies, including transitive dependencies (also known as nested dependencies). You can take advantage of dependency scanning by either: @@ -55,7 +55,7 @@ is **not** `19.03.0`. See [troubleshooting information](#error-response-from-dae ## Supported languages and package managers -GitLab relies on [`rules`](../../../ci/yaml/README.md#rules) to start relevant analyzers depending on the languages detected in the repository. +GitLab relies on [`rules`](../../../ci/yaml/index.md#rules) to start relevant analyzers depending on the languages detected in the repository. The current detection logic limits the maximum search depth to two levels. For example, the `gemnasium-dependency_scanning` job is enabled if a repository contains either a `Gemfile` or `api/Gemfile` file, but not if the only supported dependency file is `api/client/Gemfile`. The following languages and dependency managers are supported: @@ -90,7 +90,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Configuration To enable dependency scanning for GitLab 11.9 and later, you must -[include](../../../ci/yaml/README.md#includetemplate) the +[include](../../../ci/yaml/index.md#includetemplate) the [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) that is provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined @@ -106,14 +106,39 @@ include: The included template creates dependency scanning jobs in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[dependency scanning report artifact](../../../ci/yaml/README.md#artifactsreportsdependency_scanning) +[dependency scanning report artifact](../../../ci/yaml/index.md#artifactsreportsdependency_scanning) that you can later download and analyze. Due to implementation limitations, we always take the latest dependency scanning artifact available. +### Enable Dependency Scanning via an automatic merge request + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4908) in GitLab 14.1. +> - [Deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. +> - Enabled on GitLab.com. +> - Recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-configure-dependency-scanning-via-a-merge-request). **(ULTIMATE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +There can be +[risks when disabling released features](../../../user/feature_flags.md#risks-when-disabling-released-features). +Refer to this feature's version history for more details. + +To enable Dependency Scanning in a project, you can create a merge request +from the Security Configuration page. + +1. In the project where you want to enable Dependency Scanning, navigate to + **Security & Compliance > Configuration**. +1. In the **Dependency Scanning** row, select **Configure via Merge Request**. + +This automatically creates a merge request with the changes necessary to enable Dependency Scanning +that you can review and merge to complete the configuration. + ### Customizing the dependency scanning settings The dependency scanning settings can be changed through [CI/CD variables](#available-cicd-variables) by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. For example: ```yaml @@ -124,14 +149,14 @@ variables: SECURE_LOG_LEVEL: error ``` -Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline +Because template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline configuration, the last mention of the variable takes precedence. ### Overriding dependency scanning jobs WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. To override a job definition (for example, to change properties like `variables` or `dependencies`), declare a new job with the same name as the one to override. Place this new job after the template @@ -189,12 +214,12 @@ The following variables are used for configuring specific analyzers (used for a | `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | | `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | -| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`. Maven and Gradle use the Java version specified by this value (Dependency Scanning for Gradle does not currently support Java `16`). | +| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | | `PIP_INDEX_URL` | `gemnasium-python` | `https://pypi.org/simple` | Base URL of Python Package Index. | -| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma-separated. | +| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma-separated. **Warning:** Please read [the following security consideration](#python-projects) when using this environment variable. | | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | @@ -217,7 +242,7 @@ variables: -----END CERTIFICATE----- ``` -The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/README.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. +The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. ### Using private Maven repositories @@ -552,6 +577,18 @@ gemnasium-dependency_scanning: - tar -xzf gemnasium_db.tar.gz -C $GEMNASIUM_DB_LOCAL_PATH ``` +## Warnings + +### Python projects + +Extra care needs to be taken when using the [`PIP_EXTRA_INDEX_URL`](https://pipenv.pypa.io/en/latest/cli/#envvar-PIP_EXTRA_INDEX_URL) +environment variable due to a possible exploit documented by [CVE-2018-20225](https://nvd.nist.gov/vuln/detail/CVE-2018-20225): + +> An issue was discovered in pip (all versions) because it installs the version with the highest version number, even if the user had +intended to obtain a private package from a private index. This only affects use of the `PIP_EXTRA_INDEX_URL` option, and exploitation +requires that the package does not already exist in the public index (and thus the attacker can put the package there with an arbitrary +version number). + ## Limitations ### Referencing local dependencies using a path in JavaScript projects @@ -584,7 +621,7 @@ Generally, the approach is the following: 1. Define a dedicated converter job in your `.gitlab-ci.yml` file. Use a suitable Docker image, script, or both to facilitate the conversion. 1. Let that job upload the converted, supported file as an artifact. -1. Add [`dependencies: [<your-converter-job>]`](../../../ci/yaml/README.md#dependencies) +1. Add [`dependencies: [<your-converter-job>]`](../../../ci/yaml/index.md#dependencies) to your `dependency_scanning` job to make use of the converted definitions files. For example, the currently unsupported `poetry.lock` file can be @@ -631,7 +668,7 @@ For information on this, see the [general Application Security troubleshooting s ### Limitation when using rules:exists The [dependency scanning CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) -uses the [`rules:exists`](../../../ci/yaml/README.md#rulesexists) +uses the [`rules:exists`](../../../ci/yaml/index.md#rulesexists) syntax. This directive is limited to 10000 checks and always returns `true` after reaching this number. Because of this, and depending on the number of files in your repository, a dependency scanning job might be triggered even if the scanner doesn't support your project. @@ -644,3 +681,22 @@ with a dependency on this version of Python should use `retire.js` version 2.10. ### Error: `dependency_scanning is used for configuration only, and its script should not be executed` For information on this, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). + +### Enable or disable Configure Dependency Scanning via a Merge Request + +Configure Dependency Scanning via a Merge Request is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it. + +To disable it: + +```ruby +Feature.disable(:sec_dependency_scanning_ui_enable) +``` + +To enable it: + +```ruby +Feature.enable(:sec_dependency_scanning_ui_enable) +``` diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index bf812b25b5f..616d2f8c790 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -33,16 +33,17 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | Secure scanning tool | Description | |:-----------------------------------------------------------------------------|:-----------------------------------------------------------------------| -| [Container Scanning](container_scanning/index.md) **(ULTIMATE)** | Scan Docker containers for known vulnerabilities. | -| [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | -| [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | -| [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | -| [DAST API](dast_api/index.md) **(ULTIMATE)** | Analyze running web APIs for known vulnerabilities. | -| [API fuzzing](api_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities in web APIs with fuzzing. | -| [Secret Detection](secret_detection/index.md) | Analyze Git history for leaked secrets. | -| [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | -| [Static Application Security Testing (SAST)](sast/index.md) | Analyze source code for known vulnerabilities. | -| [Coverage fuzzing](coverage_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities with coverage-guided fuzzing. | +| [Container Scanning](container_scanning/index.md) **(ULTIMATE)** | Scan Docker containers for known vulnerabilities. | +| [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | +| [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | +| [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | +| [DAST API](dast_api/index.md) **(ULTIMATE)** | Analyze running web APIs for known vulnerabilities. | +| [API fuzzing](api_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities in web APIs with fuzzing. | +| [Secret Detection](secret_detection/index.md) | Analyze Git history for leaked secrets. | +| [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | +| [Static Application Security Testing (SAST)](sast/index.md) | Analyze source code for known vulnerabilities. | +| [Coverage fuzzing](coverage_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities with coverage-guided fuzzing. | +| [Cluster Image Scanning](cluster_image_scanning/index.md) **(ULTIMATE)** | Scan Kubernetes clusters for known vulnerabilities. | ## Security scanning with Auto DevOps @@ -99,7 +100,7 @@ the container-scanning analyzer which uses ### Use security scanning tools with Pipelines for Merge Requests By default, the application security jobs are configured to run for branch pipelines only. -To use them with [pipelines for merge requests](../../ci/merge_request_pipelines/index.md), +To use them with [pipelines for merge requests](../../ci/pipelines/merge_request_pipelines.md), you may need to override the default `rules:` configuration to add: ```yaml @@ -129,7 +130,7 @@ All jobs are permitted to fail by default. This means that if they fail it do no If you want to prevent vulnerabilities from being merged, you should do this by adding [Security Approvals in Merge Requests](#security-approvals-in-merge-requests) which prevents unknown, high or critical findings from being merged without an approval from a specific group of people that you choose. -We do not recommend changing the job [`allow_failure` setting](../../ci/yaml/README.md#allow_failure) as that fails the entire pipeline. +We do not recommend changing the job [`allow_failure` setting](../../ci/yaml/index.md#allow_failure) as that fails the entire pipeline. ### JSON Artifact @@ -209,7 +210,6 @@ request contains a denied license. For more details, see [Enabling license appro Prerequisites: -- At least one [security scanner job](#security-scanning-tools) must be enabled. - Maintainer or Owner [role](../permissions.md#project-members-permissions). For this approval group, you must set the number of approvals required to greater than zero. @@ -238,7 +238,7 @@ to pass a username and password. You can set it under your project's settings so that your credentials aren't exposed in `.gitlab-ci.yml`. If the username is `myuser` and the password is `verysecret` then you would -[set the following variable](../../ci/variables/README.md#custom-cicd-variables) +[set the following variable](../../ci/variables/index.md#custom-cicd-variables) under your project's settings: | Type | Key | Value | @@ -358,7 +358,7 @@ You can do it quickly by following the hyperlink given to run a new pipeline. ### Getting error message `sast job: stage parameter should be [some stage name here]` -When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +When [including](../../ci/yaml/index.md#includetemplate) a `.gitlab-ci.yml` template like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: @@ -406,12 +406,12 @@ This is often followed by the [error `No files to upload`](../../ci/pipelines/jo and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Please check the entire job log for such messages. If you don't find these messages, retry the failed job after setting `SECURE_LOG_LEVEL: "debug"` as a -[custom CI/CD variable](../../ci/variables/README.md#custom-cicd-variables). +[custom CI/CD variable](../../ci/variables/index.md#custom-cicd-variables). This provides useful information to investigate further. ### Getting error message `sast job: config key may not be used with 'rules': only/except` -When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +When [including](../../ci/yaml/index.md#includetemplate) a `.gitlab-ci.yml` template like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: @@ -422,7 +422,7 @@ Found errors in your .gitlab-ci.yml: ``` This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) -with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#only--except) +with [the deprecated `only` or `except` syntax.](../../ci/yaml/index.md#only--except) To fix this issue, you must either: - [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). @@ -433,8 +433,8 @@ To fix this issue, you must either: #### Transitioning your `only/except` syntax to `rules` When overriding the template to control job execution, previous instances of -[`only` or `except`](../../ci/yaml/README.md#only--except) are no longer compatible -and must be transitioned to [the `rules` syntax](../../ci/yaml/README.md#rules). +[`only` or `except`](../../ci/yaml/index.md#only--except) are no longer compatible +and must be transitioned to [the `rules` syntax](../../ci/yaml/index.md#rules). If your override is aimed at limiting jobs to only run on `master`, the previous syntax would look similar to: @@ -490,11 +490,11 @@ spotbugs-sast: - if: $CI_COMMIT_TAG == null ``` -[Learn more on the usage of `rules`](../../ci/yaml/README.md#rules). +[Learn more on the usage of `rules`](../../ci/yaml/index.md#rules). #### Pin your templates to the deprecated versions -To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/README.md#rules). +To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/index.md#rules). If you're unable to immediately update your CI configuration, there are several workarounds that involve pinning to the previous template versions, for example: diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 77a15a37c55..d87da15b4b0 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -108,7 +108,7 @@ example of such a transfer: ### Using the official GitLab template -GitLab provides a [vendored template](../../../ci/yaml/README.md#includetemplate) +GitLab provides a [vendored template](../../../ci/yaml/index.md#includetemplate) to ease this process. This template should be used in a new, empty project, with a `gitlab-ci.yml` file containing: @@ -149,7 +149,7 @@ GitLab.com. To do so, set the CI/CD variable `SECURE_ANALYZERS_PREFIX` with the project [container registry](../../packages/container_registry/index.md). You can set this variable in the projects' `.gitlab-ci.yml`, or -in the GitLab UI at the project or group level. See the [GitLab CI/CD variables page](../../../ci/variables/README.md#custom-cicd-variables) +in the GitLab UI at the project or group level. See the [GitLab CI/CD variables page](../../../ci/variables/index.md#custom-cicd-variables) for more information. #### Variables diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 92f0d6924b3..076872c9864 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -191,6 +191,10 @@ changes. You can always change the **Security policy project** by navigating to your project's **Security & Compliance > Policies** and modifying the selected project. +NOTE: +Only project Owners have the [permissions](../../permissions.md#project-members-permissions) +to select Security Policy Project. + ## Roadmap See the [Category Direction page](https://about.gitlab.com/direction/protect/container_network_security/) diff --git a/doc/user/application_security/sast/img/sast_results_in_mr_v14_0.png b/doc/user/application_security/sast/img/sast_results_in_mr_v14_0.png Binary files differnew file mode 100644 index 00000000000..17edc30fa0f --- /dev/null +++ b/doc/user/application_security/sast/img/sast_results_in_mr_v14_0.png diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index e80807b31bf..c64df616925 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -15,13 +15,15 @@ The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab. explains how 4 of the top 6 attacks were application based. Download it to learn how to protect your organization. -If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known -vulnerabilities using Static Application Security Testing (SAST). GitLab checks the SAST report and -compares the found vulnerabilities between the source and target branches. +If you're using [GitLab CI/CD](../../../ci/index.md), you can use Static Application Security +Testing (SAST) to check your source code for known vulnerabilities. When a pipeline completes, +the results of the SAST analysis are processed and shown in the pipeline's Security tab. If the +pipeline is associated with a merge request, the SAST analysis is compared with the results of +the target branch's analysis (if available). The results of that comparison are shown in the merge +request. **(ULTIMATE)** If the pipeline is running from the default branch, the results of the SAST +analysis are available in the [security dashboards](../security_dashboard/index.md). -Details of the vulnerabilities found are included in the merge request. **(ULTIMATE)** - - + The results are sorted by the priority of the vulnerability: @@ -160,7 +162,7 @@ To configure SAST for a project you can: ### Configure SAST manually -For GitLab 11.9 and later, to enable SAST you must [include](../../../ci/yaml/README.md#includetemplate) +For GitLab 11.9 and later, to enable SAST you must [include](../../../ci/yaml/index.md#includetemplate) the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined that template. @@ -176,7 +178,7 @@ The included template creates SAST jobs in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[SAST report artifact](../../../ci/yaml/README.md#artifactsreportssast) +[SAST report artifact](../../../ci/yaml/index.md#artifactsreportssast) that you can later download and analyze. Due to implementation limitations, we always take the latest SAST artifact available. @@ -204,7 +206,7 @@ page: The SAST settings can be changed through [CI/CD variables](#available-cicd-variables) by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. In the following example, we include the SAST template and at the same time we set the `SAST_GOSEC_LEVEL` variable to `2`: @@ -216,14 +218,14 @@ variables: SAST_GOSEC_LEVEL: 2 ``` -Because the template is [evaluated before](../../../ci/yaml/README.md#include) +Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline configuration, the last mention of the variable takes precedence. ### Overriding SAST jobs WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. To override a job definition, (for example, change properties like `variables` or `dependencies`), declare a job with the same name as the SAST job to override. Place this new job after the template @@ -463,7 +465,7 @@ variables: -----END CERTIFICATE----- ``` -The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/README.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. +The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. #### Docker images @@ -497,7 +499,7 @@ Some analyzers can be customized with CI/CD variables. | `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | | `KUBESEC_HELM_CHARTS_PATH` | Kubesec | Optional path to Helm charts that `helm` uses to generate a Kubernetes manifest that `kubesec` scans. If dependencies are defined, `helm dependency build` should be ran in a `before_script` to fetch the necessary dependencies. | | `KUBESEC_HELM_OPTIONS` | Kubesec | Additional arguments for the `helm` executable. | -| `COMPILE` | SpotBugs | Set to `false` to disable project compilation and dependency fetching. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195252) in GitLab 13.1. | +| `COMPILE` | Gosec, SpotBugs | Set to `false` to disable project compilation and dependency fetching. [Introduced for `SpotBugs`](https://gitlab.com/gitlab-org/gitlab/-/issues/195252) analyzer in GitLab 13.1 and [`Gosec`](https://gitlab.com/gitlab-org/gitlab/-/issues/330678) analyzer in GitLab 14.0. | | `ANT_HOME` | SpotBugs | The `ANT_HOME` variable. | | `ANT_PATH` | SpotBugs | Path to the `ant` executable. | | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | @@ -510,8 +512,8 @@ Some analyzers can be customized with CI/CD variables. | `SBT_PATH` | SpotBugs | Path to the `sbt` executable. | | `FAIL_NEVER` | SpotBugs | Set to `1` to ignore compilation failure. | | `SAST_GOSEC_CONFIG` | Gosec | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328301)** in GitLab 14.0 - use custom rulesets instead. Path to configuration for Gosec (optional). | -| `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | -| `SAST_DISABLE_BABEL` | NodeJsScan | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64025)** in GitLab 13.5 | +| `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | +| `SAST_DISABLE_BABEL` | NodeJsScan | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64025)** in GitLab 13.5 | | `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://r2c.dev/). Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330565) in GitLab 14.0. | #### Custom CI/CD variables @@ -519,7 +521,7 @@ Some analyzers can be customized with CI/CD variables. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18193) in GitLab Ultimate 12.5. In addition to the aforementioned SAST configuration CI/CD variables, -all [custom variables](../../../ci/variables/README.md#custom-cicd-variables) are propagated +all [custom variables](../../../ci/variables/index.md#custom-cicd-variables) are propagated to the underlying SAST analyzer images if [the SAST vendored template](#configuration) is used. @@ -554,7 +556,7 @@ The SAST tool emits a JSON report file. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). The JSON report file can be downloaded from the CI pipelines page, or the -pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/README.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). +pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). Here's an example SAST report: @@ -763,7 +765,7 @@ uses the `rules:exists` parameter. For performance reasons, a maximum number of against the given glob pattern. If the number of matches exceeds the maximum, the `rules:exists` parameter returns `true`. Depending on the number of files in your repository, a SAST job might be triggered even if the scanner doesn't support your project. For more details about this issue, see -the [`rules:exists` documentation](../../../ci/yaml/README.md#rulesexists). +the [`rules:exists` documentation](../../../ci/yaml/index.md#rulesexists). ### SpotBugs UTF-8 unmappable character errors @@ -789,7 +791,7 @@ For Maven builds, add the following to your `pom.xml` file: ### Flawfinder encoding error -This occurs when Flawfinder encounters an invalid UTF-8 character. To fix this, convert all source code in your project to UTF-8 character encoding. This can be done with [`cvt2utf`](https://github.com/x1angli/cvt2utf) or [`iconv`](https://www.gnu.org/software/libiconv/documentation/libiconv-1.13/iconv.1.html) either over the entire project or per job using the [`before_script`](../../../ci/yaml/README.md#before_script) feature. +This occurs when Flawfinder encounters an invalid UTF-8 character. To fix this, convert all source code in your project to UTF-8 character encoding. This can be done with [`cvt2utf`](https://github.com/x1angli/cvt2utf) or [`iconv`](https://www.gnu.org/software/libiconv/documentation/libiconv-1.13/iconv.1.html) either over the entire project or per job using the [`before_script`](../../../ci/yaml/index.md#before_script) feature. ### Semgrep slowness, unexpected results, or other errors diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index f4aa9dc2787..938bd3b41d5 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -133,24 +133,14 @@ The included template creates Secret Detection jobs in your CI/CD pipeline and s your project's source code for secrets. The results are saved as a -[Secret Detection report artifact](../../../ci/yaml/README.md#artifactsreportssecret_detection) +[Secret Detection report artifact](../../../ci/yaml/index.md#artifactsreportssecret_detection) that you can later download and analyze. Due to implementation limitations, we always take the latest Secret Detection artifact available. ### Enable Secret Detection via an automatic merge request **(ULTIMATE SELF)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4496) in GitLab 13.11. -> - [Deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-configure-secret-detection-via-a-merge-request). **(ULTIMATE SELF)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -There can be -[risks when disabling released features](../../../user/feature_flags.md#risks-when-disabling-released-features). -Refer to this feature's version history for more details. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4496) in GitLab 13.11, behind a feature flag, enabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/329886) in GitLab 14.1. To enable Secret Detection in a project, you can create a merge request from the Security Configuration page. @@ -166,15 +156,15 @@ that you can review and merge to complete the configuration. The Secret Detection scan settings can be changed through [CI/CD variables](#available-cicd-variables) by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. To override a job definition, (for example, change properties like `variables` or `dependencies`), declare a job with the same name as the SAST job to override. Place this new job after the template inclusion and specify any additional keys under it. WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. #### GIT_DEPTH @@ -197,7 +187,7 @@ secret_detection: SECRET_DETECTION_HISTORIC_SCAN: "true" ``` -Because the template is [evaluated before](../../../ci/yaml/README.md#include) +Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline configuration, the last mention of the variable takes precedence. #### Available CI/CD variables @@ -285,7 +275,7 @@ Post-processing is currently limited to a project's default branch, see the abov sequenceDiagram autonumber Rails->>+Sidekiq: gl-secret-detection-report.json - Sidekiq-->+Sidekiq: BuildFinishedWorker + Sidekiq-->+Sidekiq: Ci::BuildFinishedWorker Sidekiq-->+RevocationAPI: GET revocable keys types RevocationAPI-->>-Sidekiq: OK Sidekiq->>+RevocationAPI: POST revoke revocable keys @@ -360,6 +350,21 @@ Support for custom certificate authorities was introduced in the following versi | -------- | ------- | | secrets | [v3.0.0](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/releases/v3.0.0) | +To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle +of CA certs that you want to trust in the SAST environment. The `ADDITIONAL_CA_CERT_BUNDLE` value should contain the [text representation of the X.509 PEM public-key certificate](https://tools.ietf.org/html/rfc7468#section-5.1). For example, to configure this value in the `.gitlab-ci.yml` file, use the following: + +```yaml +variables: + ADDITIONAL_CA_CERT_BUNDLE: | + -----BEGIN CERTIFICATE----- + MIIGqTCCBJGgAwIBAgIQI7AVxxVwg2kch4d56XNdDjANBgkqhkiG9w0BAQsFADCB + ... + jWgmPqF3vUbZE0EyScetPJquRFRKIesyJuBFMAs= + -----END CERTIFICATE----- +``` + +The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. + ### Set Secret Detection CI/CD variables to use local Secret Detection analyzer Add the following configuration to your `.gitlab-ci.yml` file. You must replace @@ -385,7 +390,7 @@ For information on this, see the [general Application Security troubleshooting s ### Error: `Couldn't run the gitleaks command: exit status 2` If a pipeline is triggered from a Merge Request containing 60 commits while the `GIT_DEPTH` variable -is set to 50 (a [project default](../../../ci/pipelines/settings.md#git-shallow-clone)), +is set to 50 (a [project default](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone)), the Secret Detection job fails as the clone is not deep enough to contain all of the relevant commits. @@ -409,22 +414,3 @@ secret_detection: variables: GIT_DEPTH: 100 ``` - -### Enable or disable Configure Secret Detection via a Merge Request - -Configure Secret Detection via a Merge Request is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:sec_secret_detection_ui_enable) -``` - -To disable it: - -```ruby -Feature.disable(:sec_secret_detection_ui_enable) -``` diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 01de066367c..806bc03e30e 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -33,13 +33,14 @@ The security dashboard and vulnerability report displays information about vulne - [Dynamic Application Security Testing](../dast/index.md) - [Dependency Scanning](../dependency_scanning/index.md) - [Static Application Security Testing](../sast/index.md) +- [Cluster Image Scanning](../cluster_image_scanning/index.md) - And [others](../index.md#security-scanning-tools)! ## Prerequisites 1. At least one project inside a group must be configured with at least one of the [supported reports](#supported-reports). -1. The configured jobs must use the [new `reports` syntax](../../../ci/yaml/README.md#artifactsreports). +1. The configured jobs must use the [new `reports` syntax](../../../ci/yaml/index.md#artifactsreports). 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used. If you're using the shared runners on GitLab.com, this is already the case. @@ -76,7 +77,7 @@ CSV file containing details of the resources scanned. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285477) in GitLab 13.11, date range slider to visualize data between given dates. A project's Security Dashboard displays a chart with the total number of vulnerabilities -over time. It updates daily with up to 365 days of historical data. By default, +over time with up to 365 days of historical data. Data is refreshed daily at 1:15am UTC. By default, it shows statistics for all vulnerability severities. To access the dashboard, from your project's home page go to **Security & Compliance > Security Dashboard**. diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index ce30accfb4d..c96497e9233 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -118,6 +118,7 @@ The type of scan. This must be one of the following: - `dependency_scanning` - `dast` - `sast` +- `cluster_image_scanning` ### Scanner diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index e1200c60419..8cecb9c5929 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -27,20 +27,19 @@ your application's Kubernetes namespace. This section has the following prerequisites: - Your project contains at least one [environment](../../../ci/environments/index.md) -- You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd) +- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium) - You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration) If you're using custom Helm values for Cilium, you must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Cilium values](../../clusters/applications.md#install-cilium-using-gitlab-cicd): +your [Cilium values](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium): ```yaml -global: - hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' +hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' ``` The **Container Network Policy** section displays the following information @@ -54,7 +53,11 @@ about your packet flow: If a significant percentage of packets is dropped, you should investigate it for potential threats by -[examining the Cilium logs](../../clusters/applications.md#install-cilium-using-gitlab-cicd). +examining the Cilium logs: + +```shell +kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor +``` ## Container Network Policy management @@ -67,7 +70,7 @@ status, and create and edit deployed policies. This section has the following prerequisites: - Your project contains at least one [environment](../../../ci/environments/index.md) -- You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd) +- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium) Network policies are fetched directly from the selected environment's deployment platform. Changes performed outside of this tab are diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 9866709bacc..6437f2325e8 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -164,7 +164,7 @@ The following vulnerability scanners and their databases are regularly updated: |:----------------------------------------------------------------|----------------------------------| | [Container Scanning](../container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. | | [Dependency Scanning](../dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | -| [Dynamic Application Security Testing (DAST)](../dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/master/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | +| [Dynamic Application Security Testing (DAST)](../dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | | [Static Application Security Testing (SAST)](../sast/index.md) | Relies exclusively on [the tools GitLab wraps](../sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | You do not have to update GitLab to benefit from the latest vulnerabilities definitions. diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 07025d193af..da59c0fbe79 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -128,11 +128,11 @@ To view the relevant file, select the filename in the vulnerability's details. ## View issues raised for a vulnerability The **Activity** column indicates the number of issues that have been created for the vulnerability. -Hover over an **Activity** entry and select a link go to that issue. The status of whether the issue is open or closed also displays in the hover menu. +Hover over an **Activity** entry and select a link go to that issue. The status of whether the issue is open or closed also displays in the hover menu.  -If Jira issue support is enabled, the issue link found in the Activity entry links out to the issue in Jira. Unlike GitLab issues, the status of whether a Jira issue is Open or Closed does not display in the GitLab UI. +If Jira issue support is enabled, the issue link found in the Activity entry links out to the issue in Jira. Unlike GitLab issues, the status of whether a Jira issue is Open or Closed does not display in the GitLab UI. ## Change status of vulnerabilities diff --git a/doc/user/clusters/agent/ci_cd_tunnel.md b/doc/user/clusters/agent/ci_cd_tunnel.md new file mode 100644 index 00000000000..1f794bac37f --- /dev/null +++ b/doc/user/clusters/agent/ci_cd_tunnel.md @@ -0,0 +1,67 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# CI/CD Tunnel + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. + +The CI/CD Tunnel enables users to access Kubernetes clusters from GitLab CI/CD jobs even if there is no network +connectivity between GitLab Runner and a cluster. GitLab Runner does not have to be running in the same cluster. + +Only CI/CD jobs set in the configuration project can access one of the configured agents. + +Prerequisites: + +- A running [`kas` instance](index.md#set-up-the-kubernetes-agent-server). +- A [configuration repository](index.md#define-a-configuration-repository) with an Agent config file + installed (`.gitlab/agents/<agent-name>/config.yaml`). +- An [Agent record](index.md#create-an-agent-record-in-gitlab). +- The agent is [installed in the cluster](index.md#install-the-agent-into-the-cluster). + +To access your cluster from a CI/CD job through the tunnel: + +1. In your `.gitlab-ci.yml` add a section that creates a `kubectl` compatible configuration file (`kubecontext`) and use it in one + or more jobs: + + ```yaml + variables: + AGENT_ID: 4 # agent id that you got when you created the agent record + KUBE_CFG_FILE: "$CI_PROJECT_DIR/.kubeconfig.agent.yaml" + + .kubectl_config: &kubectl_config + - | + cat << EOF > "$KUBE_CFG_FILE" + apiVersion: v1 + kind: Config + clusters: + - name: agent + cluster: + server: https://kas.gitlab.com/k8s-proxy/ + users: + - name: agent + user: + token: "ci:$AGENT_ID:$CI_JOB_TOKEN" + contexts: + - name: agent + context: + cluster: agent + user: agent + current-context: agent + EOF + + deploy: + image: + name: bitnami/kubectl:latest + entrypoint: [""] + script: + - *kubectl_config + - kubectl --kubeconfig="$KUBE_CFG_FILE" get pods + ``` + +1. Execute `kubectl` commands directly against your cluster with this CI/CD job you just created. + +We are working on [creating the configuration file automatically](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) +to simplify the process. diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 414ed8377db..83933524fd4 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -7,7 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Kubernetes Agent **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. -> - [In GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/300960), KAS became available on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program. +> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/7) in GitLab 13.6, `grpcs` is supported. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300960) in GitLab 13.10, KAS became available on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program. > - Introduced in GitLab 13.11, the GitLab Kubernetes Agent became available to every project on GitLab.com. The [GitLab Kubernetes Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) @@ -16,13 +17,14 @@ tasks in a secure and cloud-native way. It enables: - Integrating GitLab with a Kubernetes cluster behind a firewall or NAT (network address translation). -- Pull-based GitOps deployments by leveraging the - [GitOps Engine](https://github.com/argoproj/gitops-engine). +- Pull-based GitOps deployments. +- [Inventory object](../../infrastructure/clusters/deploy/inventory_object.md) to keep track of objects applied to your cluster. - Real-time access to API endpoints in a cluster. - Alert generation based on [Container network policy](../../application_security/threat_monitoring/index.md#container-network-policy). +- [CI/CD Tunnel](ci_cd_tunnel.md) that enables users to access Kubernetes clusters from GitLab CI/CD jobs even if there is no network connectivity between GitLab Runner and a cluster. Many more features are planned. Please review [our roadmap](https://gitlab.com/groups/gitlab-org/-/epics/3329) -and [our development documentation](../../../development/agent/index.md). +and [our development documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/doc). ## GitLab Agent GitOps workflow @@ -37,7 +39,9 @@ sequenceDiagram participant M as Manifest repository participant K as Kubernetes Agent participant C as Agent configuration repository - K->C: Grab the configuration + loop Regularly + K-->>C: Grab the configuration + end D->>+A: Pushing code changes A->>M: Updating manifest loop Regularly @@ -246,12 +250,11 @@ example [`resources.yml` file](#example-resourcesyml-file) in the following ways - Specify the `grpc` scheme if both Agent and Server are installed in one cluster. In this case, you may specify `kas-address` value as `grpc://gitlab-kas.<your-namespace>:8150`) to use gRPC directly, where `gitlab-kas` - is the name of the service created by `gitlab-kas` chart, and `your-namespace` - is the namespace where the chart was installed. Encrypted gRPC is not supported yet. - Follow the - [Support TLS for gRPC communication issue](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/7) - for progress updates. - - When deploying KAS through the [GitLab chart](https://docs.gitlab.com/charts/), it's possible to customize the `kas-address` for `wss` and `ws` schemes to whatever you need. + is the name of the service created by `gitlab-kas` chart, and `<your-namespace>` + is the namespace where the chart was installed. + - Specify the `grpcs` scheme to use an encrypted gRPC connection. + - When deploying KAS through the [GitLab chart](https://docs.gitlab.com/charts/), it's possible to customize the + `kas-address` for `wss` and `ws` schemes to whatever you need. Check the [chart's KAS Ingress documentation](https://docs.gitlab.com/charts/charts/gitlab/kas/#ingress) to learn more about it. - In the near future, Omnibus GitLab intends to provision `gitlab-kas` under a sub-domain by default, instead of the `/-/kubernetes-agent/` path. Please follow [this issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5784) for details. @@ -447,42 +450,21 @@ There are several components that work in concert for the Agent to generate the - A working Kubernetes cluster. - Cilium integration through either of these options: - - Installation through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd). + - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an existing installation. - One or more network policies through any of these options: - Use the [Container Network Policy editor](../../application_security/threat_monitoring/index.md#container-network-policy-editor) to create and manage policies. - Use an [AutoDevOps](../../application_security/threat_monitoring/index.md#container-network-policy-management) configuration. - Add the required labels and annotations to existing network policies. -- Use a configuration repository to inform the Agent through a `config.yaml` file, which - repositories can synchronize with. This repository might be the same, or a separate GitLab - project. +- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) The setup process follows the same steps as [GitOps](#get-started-with-gitops-and-the-gitlab-agent), with the following differences: -- When you define a configuration repository, you must do so with [Cilium settings](#define-a-configuration-repository-with-cilium-settings). +- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). - You do not need to specify the `gitops` configuration section. -### Define a configuration repository with Cilium settings - -You need a GitLab repository to contain your Agent configuration. The minimal repository layout -looks like this: - -```plaintext -.gitlab/agents/<agent-name>/config.yaml -``` - -Your `config.yaml` file must specify the `host` and `port` of your Hubble Relay service. If your -Cilium integration was performed through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd), -you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80`: - -```yaml -cilium: - hubble_relay_address: "<hubble-relay-host>:<hubble-relay-port>" - ... -``` - ## Management interfaces Users with at least the [Developer](../../permissions.md) can access the user interface @@ -516,9 +498,17 @@ This error is shown if there are some connectivity issues between the address specified as `kas-address`, and your Agent pod. To fix it, make sure that you specified the `kas-address` correctly. +```json +{"level":"error","time":"2021-06-25T21:15:45.335Z","msg":"Reverse tunnel","mod_name":"reverse_tunnel","error":"Connect(): rpc error: code = Unavailable desc = connection error: desc= \"transport: Error while dialing failed to WebSocket dial: expected handshake response status code 101 but got 301\""} +``` + +This error occurs if the `kas-address` doesn't include a trailing slash. To fix it, make sure that the +`wss` or `ws` URL ends with a training slash, such as `wss://GitLab.host.tld:443/-/kubernetes-agent/` +or `ws://GitLab.host.tld:80/-/kubernetes-agent/`. + ### Agent logs - ValidationError(Deployment.metadata) -```plaintext +```json {"level":"info","time":"2020-10-30T08:56:54.329Z","msg":"Synced","project_id":"root/kas-manifest001","resource_key":"apps/Deployment/kas-test001/nginx-deployment","sync_result":"error validating data: [ValidationError(Deployment.metadata): unknown field \"replicas\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta, ValidationError(Deployment.metadata): unknown field \"selector\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta, ValidationError(Deployment.metadata): unknown field \"template\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta]"} ``` diff --git a/doc/user/clusters/agent/repository.md b/doc/user/clusters/agent/repository.md index cd40cc6810e..a3a3e4c29b0 100644 --- a/doc/user/clusters/agent/repository.md +++ b/doc/user/clusters/agent/repository.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.7. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the Kubernetes Agent became available on GitLab.com. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -51,6 +51,7 @@ gitops: # in YAML or JSON format. - id: gitlab-org/cluster-integration/gitlab-agent # Namespace to use if not set explicitly in object manifest. + # Also used for inventory ConfigMap objects. default_namespace: my-ns # Paths inside of the repository to scan for manifest files. # Directories with names starting with a dot are ignored. @@ -84,13 +85,13 @@ gitops: # https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470 # Can be: orphan, background, foreground prune_propagation_policy: foreground # 'foreground' by default - # InventoryPolicy defines if an inventory object can take over + # Inventory policy defines if an inventory object can take over # objects that belong to another inventory object or don't # belong to any inventory object. # This is done by determining if the apply/prune operation # can go through for a resource based on the comparison # the inventory-id value in the package and the owning-inventory - # annotation in the live object. + # annotation (config.k8s.io/owning-inventory) in the live object. # https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66 # Can be: must_match, adopt_if_no_inventory, adopt_all inventory_policy: must_match # 'must_match' by default @@ -157,7 +158,9 @@ cilium: hubble_relay_address: "<hubble-relay-host>:<hubble-relay-port>" ``` -If your Cilium integration was performed through GitLab Managed Apps, you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the address: +If your Cilium integration was performed through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd) or the +[cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium), +you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the address: ```yaml cilium: diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index a6aa4e00005..5d247f04d3b 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -90,7 +90,7 @@ To install applications using GitLab CI/CD: 1. Optionally, define `.gitlab/managed-apps/<application>/values.yaml` file to customize values for the installed application. -A GitLab CI/CD pipeline runs on the `master` branch to install the +A GitLab CI/CD pipeline runs on the default branch to install the applications you have configured. In case of pipeline failure, the output of the [Helm Tiller](https://v2.helm.sh/docs/install/#running-tiller-locally) binary is saved as a [CI job artifact](../../ci/pipelines/job_artifacts.md). @@ -391,16 +391,16 @@ For GitLab Runner to function, you _must_ specify the following: - `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) to register the Runner against. - `runnerRegistrationToken`: The registration token for adding new runners to GitLab. - This must be [retrieved from your GitLab instance](../../ci/runners/README.md). + This must be [retrieved from your GitLab instance](../../ci/runners/index.md). -These values can be specified using [CI/CD variables](../../ci/variables/README.md): +These values can be specified using [CI/CD variables](../../ci/variables/index.md): - `GITLAB_RUNNER_GITLAB_URL` is used for `gitlabUrl`. - `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` The methods of specifying these values are mutually exclusive. Either specify variables `GITLAB_RUNNER_REGISTRATION_TOKEN` and `GITLAB_RUNNER_TOKEN` as CI variables (recommended) or provide values for `runnerRegistrationToken:` and `runnerToken:` in `.gitlab/managed-apps/gitlab-runner/values.yaml`. If you choose to use CI variables, comment out or remove `runnerRegistrationToken:` and `runnerToken:` from `.gitlab/managed-apps/gitlab-runner/values`. -The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../ci/variables/README.md#protect-a-cicd-variable) and [masked variable](../../ci/variables/README.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml` file. +The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../ci/variables/index.md#protect-a-cicd-variable) and [masked variable](../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml` file. You can customize the installation of GitLab Runner by defining `.gitlab/managed-apps/gitlab-runner/values.yaml` file in your cluster @@ -440,8 +440,8 @@ cilium: The `clusterType` variable enables the recommended Helm variables for a corresponding cluster type. You can check the recommended variables for each cluster type in the official documentation: -- [Google GKE](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/#deploy-cilium) -- [AWS EKS](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-eks/#deploy-cilium) +- [Google GKE](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-gke/#deploy-cilium) +- [AWS EKS](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-eks/#deploy-cilium) Do not use `clusterType` for sandbox environments like [Minikube](https://minikube.sigs.k8s.io/docs/). @@ -460,7 +460,7 @@ You can check Cilium's installation status on the cluster management page: WARNING: Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/#restart-unmanaged-pods) +[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) of all affected pods in all namespaces to ensure that they are [managed](https://docs.cilium.io/en/v1.8/operations/troubleshooting/#ensure-managed-pod) by the correct networking plugin. Whenever Hubble is enabled, its related pod might require a @@ -762,7 +762,7 @@ Set: - "Redirect URI" to `http://<JupyterHub Host>/hub/oauth_callback`. - "Scope" to `api read_repository write_repository`. -In addition, the following variables must be specified using [CI/CD variables](../../ci/variables/README.md): +In addition, the following variables must be specified using [CI/CD variables](../../ci/variables/index.md): - `JUPYTERHUB_PROXY_SECRET_TOKEN` - Secure string used for signing communications from the hub. Read [`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#proxy-secrettoken). diff --git a/doc/user/clusters/integrations.md b/doc/user/clusters/integrations.md index 6c87efaf5a3..d2861ce2946 100644 --- a/doc/user/clusters/integrations.md +++ b/doc/user/clusters/integrations.md @@ -88,7 +88,7 @@ To enable the Prometheus integration for your cluster: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61077) in GitLab 13.12. You can integrate your cluster with [Elastic -Stack](https://www.elastic.co/elastic-stack) to index and [query your pod +Stack](https://www.elastic.co/elastic-stack/) to index and [query your pod logs](../project/clusters/kubernetes_pod_logs.md). ### Elastic Stack Prerequisites @@ -119,7 +119,7 @@ wget https://gitlab.com/gitlab-org/project-templates/cluster-management/-/raw/ma helm repo add gitlab https://charts.gitlab.io # Install Elastic Stack -helm install prometheus gitlab/elastic-stack -n gitlab-managed-apps --values values.yaml +helm install elastic-stack gitlab/elastic-stack -n gitlab-managed-apps --values values.yaml ``` ### Enable Elastic Stack integration for your cluster @@ -134,6 +134,6 @@ To enable the Elastic Stack integration for your cluster: - For an [instance-level cluster](../instance/clusters/index.md), navigate to your instance's **Kubernetes** page. 1. Select the **Integrations** tab. -1. Check the **Enable Prometheus integration** checkbox. +1. Check the **Enable Elastic Stack integration** checkbox. 1. Click **Save changes**. 1. Go to the **Health** tab to see your cluster's metrics. diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index f741ab2d95a..204afa9fc89 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -22,7 +22,7 @@ This can be useful for: ## Permissions Only the management project receives `cluster-admin` privileges. All -other projects continue to receive [namespace scoped `edit` level privileges](../project/clusters/add_remove_clusters.md#rbac-cluster-resources). +other projects continue to receive [namespace scoped `edit` level privileges](../project/clusters/cluster_access.md#rbac-cluster-resources). Management projects are restricted to the following: @@ -45,12 +45,11 @@ To use a cluster management project for a cluster: To select a cluster management project to use: 1. Navigate to the appropriate configuration page. For a: - - [Project-level cluster](../project/clusters/index.md), navigate to your project's + - [Project-level cluster](../project/clusters/index.md), go to your project's **Infrastructure > Kubernetes clusters** page. - - [Group-level cluster](../group/clusters/index.md), navigate to your group's **Kubernetes** - page. - - [Instance-level cluster](../instance/clusters/index.md), navigate to Admin Area's **Kubernetes** + - [Group-level cluster](../group/clusters/index.md), go to your group's **Kubernetes** page. + - [Instance-level cluster](../instance/clusters/index.md), go to **Menu >** **{admin}** **Admin > Kubernetes** page. 1. Select the project using **Cluster management project field** in the **Advanced settings** section. @@ -59,7 +58,7 @@ To select a cluster management project to use: ### Configuring your pipeline After designating a project as the management project for the cluster, -write a [`.gitlab-ci.yml`](../../ci/yaml/README.md) in that project. For example: +write a [`.gitlab-ci.yml`](../../ci/yaml/index.md) in that project. For example: ```yaml configure cluster: @@ -72,7 +71,7 @@ configure cluster: ### Setting the environment scope [Environment -scopes](../project/clusters/index.md#setting-the-environment-scope) +scopes](../project/clusters/multiple_kubernetes_clusters.md#setting-the-environment-scope) are usable when associating multiple clusters to the same management project. @@ -88,7 +87,7 @@ to a management project: | Production | `production` | The following environments set in -[`.gitlab-ci.yml`](../../ci/yaml/README.md) deploy to the +[`.gitlab-ci.yml`](../../ci/yaml/index.md) deploy to the Development, Staging, and Production cluster respectively. ```yaml diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index 52390cb18b0..eb86c1702cc 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -18,10 +18,10 @@ need, or even add new ones that are not built-in. ## How to use this template -1. Create a new project, choosing "GitLab Cluster Management" from the list of [built-in project templates](../project/working_with_projects.md#built-in-templates). +1. Create a new project, choosing "GitLab Cluster Management" from the list of [built-in project templates](../project/working_with_projects.md#built-in-templates). 1. Make this project a [cluster management project](management_project.md). 1. If you used the [GitLab Managed Apps](applications.md), refer to - [Migrating from GitLab Manged Apps](migrating_from_gma_to_project_template.md). + [Migrating from GitLab Managed Apps](migrating_from_gma_to_project_template.md). ### Components @@ -37,10 +37,10 @@ In the repository of the newly-created project, you will find: The base image used in your pipeline is built by the [cluster-applications](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications) project. This image consists of a set of Bash utility scripts to support [Helm v3 releases](https://helm.sh/docs/intro/using_helm/#three-big-concepts): -- `gl-fail-if-helm2-releases-exist {namespace}`: It tries to detect whether you have apps deployed through Helm v2 - releases for a given namespace. If so, it will fail the pipeline and ask you to manually +- `gl-fail-if-helm2-releases-exist {namespace}`: It tries to detect whether you have apps deployed through Helm v2 + releases for a given namespace. If so, it will fail the pipeline and ask you to manually [migrate your Helm v2 releases to Helm v3](https://helm.sh/docs/topics/v2_v3_migration/). -- `gl-ensure-namespace {namespace}`: It creates the given namespace if it does not exist and adds the necessary label +- `gl-ensure-namespace {namespace}`: It creates the given namespace if it does not exist and adds the necessary label for the [Cilium](https://github.com/cilium/cilium/) app network policies to work. - `gl-adopt-resource-with-helm-v3 {arguments}`: Used only internally in the [cert-manager's](https://cert-manager.io/) Helmfile to facilitate the GitLab Managed Apps adoption. @@ -50,7 +50,7 @@ project. This image consists of a set of Bash utility scripts to support [Helm v #### The main `helmfile.yml` file -This file has a list of paths to other Helmfiles for each app. They're all commented out by default, so you must uncomment +This file has a list of paths to other Helmfiles for each app. They're all commented out by default, so you must uncomment the paths for the apps that you would like to manage. By default, each `helmfile.yaml` in these sub-paths will have the attribute `installed: true`, which signifies that everytime @@ -58,7 +58,7 @@ the pipeline runs, Helmfile will try to either install or update your apps accor cluster and Helm releases. If you change this attribute to `installed: false`, Helmfile will try to uninstall this app from your cluster. [Read more](https://github.com/roboll/helmfile) about how Helmfile works. -Furthermore, each app has an `applications/{app}/values.yaml` file. This is the +Furthermore, each app has an `applications/{app}/values.yaml` file. This is the place where you can define some default values for your app's Helm chart. Some apps will already have defaults pre-defined by GitLab. @@ -82,5 +82,5 @@ The [built-in supported applications](https://gitlab.com/gitlab-org/project-temp ### Migrating from GitLab Managed Apps -If you had GitLab Managed Apps, either One-Click or CI/CD install, read the docs on how to +If you had GitLab Managed Apps, either One-Click or CI/CD install, read the docs on how to [migrate from GitLab Managed Apps to project template](migrating_from_gma_to_project_template.md) diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index 7fa6ccea433..dc16cf5cc45 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -35,9 +35,9 @@ The [GitLab Managed Apps](applications.md) are deprecated in GitLab 14.0. To mig applications that you would like to manage with this project. Although you could uncomment all the ones you want to managed at once, we recommend you repeat the following steps separately for each app, so you do not get lost during the process. -1. Edit the associated `applications/{app}/helmfiles.yaml` to match the chart version currently deployed +1. Edit the associated `applications/{app}/helmfiles.yaml` to match the chart version currently deployed for your app. Take a GitLab Runner Helm v3 release as an example: - + The following command lists the releases and their versions: ```shell @@ -83,7 +83,7 @@ The [GitLab Managed Apps](applications.md) are deprecated in GitLab 14.0. To mig 1. After following all the previous steps, [run a pipeline manually](../../ci/pipelines/index.md#run-a-pipeline-manually) and watch the `apply` job logs to see if any of your applications were successfully detected, installed, and whether they got any unexpected updates. - + Some annotation checksums are expected to be updated, as well as this attribute: ```diff diff --git a/doc/user/compliance/compliance_dashboard/index.md b/doc/user/compliance/compliance_dashboard/index.md index 21af6387f9d..fb6b3fe2cf6 100644 --- a/doc/user/compliance/compliance_dashboard/index.md +++ b/doc/user/compliance/compliance_dashboard/index.md @@ -22,6 +22,22 @@ To access the Compliance Dashboard for a group, navigate to **{shield}** **Secur NOTE: The Compliance Dashboard shows only the latest MR on each project. +## Merge request drawer + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299357) in GitLab 14.1. + +When you click on a row, a drawer is shown that provides further details about the merge +request: + +- Project name and [compliance framework label](../../project/settings/index.md#compliance-frameworks), + if the project has one assigned. +- Link to the merge request. +- The merge request's branch path in the format `[source] into [target]`. +- A list of users that committed changes to the merge request. +- A list of users that commented on the merge request. +- A list of users that approved the merge request. +- The user that merged the merge request. + ## Use cases This feature is for people who care about the compliance status of projects within their group. diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png Binary files differdeleted file mode 100644 index ee3bb310c1a..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v14_2.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v14_2.png Binary files differnew file mode 100644 index 00000000000..c188c6cd834 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v14_2.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index f757a548aee..1a43c5ae96f 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5483) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. -If you're using [GitLab CI/CD](../../../ci/README.md), you can use License Compliance to search your +If you're using [GitLab CI/CD](../../../ci/index.md), you can use License Compliance to search your project's dependencies for their licenses. You can then decide whether to allow or deny the use of each license. For example, if your application uses an external (open source) library whose license is incompatible with yours, then you can deny the use of that license. @@ -47,8 +47,7 @@ When GitLab detects a **Denied** license, you can view it in the [license list](  You can view and modify existing policies from the [policies](#policies) tab. - - + ## Supported languages and package managers @@ -58,7 +57,7 @@ Java 8 and Gradle 1.x projects are not supported. The minimum supported version | Language | Package managers | Notes | |------------|----------------------------------------------------------------------------------------------|-------| -| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/) | | +| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/) (7 and earlier) | | | Go | [Godep](https://github.com/tools/godep), [go mod](https://github.com/golang/go/wiki/Modules) | | | Java | [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) | | | .NET | [NuGet](https://www.nuget.org/) | The .NET Framework is supported via the [mono project](https://www.mono-project.com/). There are, however, some limitations. The scanner doesn't support Windows-specific dependencies and doesn't report dependencies of your project's listed dependencies. Also, the scanner always marks detected licenses for all dependencies as `unknown`. | @@ -74,12 +73,12 @@ The reported licenses might be incomplete or inaccurate. |------------|---------------------------------------------------------------------------------------------------------------| | JavaScript | [Yarn](https://yarnpkg.com/) | | Go | `go get`, `gvt`, `glide`, `dep`, `trash`, `govendor` | -| Erlang | [Rebar](https://www.rebar3.org/) | +| Erlang | [Rebar](https://rebar3.org/) | | Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage), [CocoaPods](https://cocoapods.org/) v0.39 and below | | Elixir | [Mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) | | C++/C | [Conan](https://conan.io/) | | Scala | [sbt](https://www.scala-sbt.org/) | -| Rust | [Cargo](https://crates.io) | +| Rust | [Cargo](https://crates.io) | | PHP | [Composer](https://getcomposer.org/) | ## Requirements @@ -90,11 +89,11 @@ To run a License Compliance scanning job, you need GitLab Runner with the ## Configuration For GitLab 12.8 and later, to enable License Compliance, you must -[include](../../../ci/yaml/README.md#includetemplate) the +[include](../../../ci/yaml/index.md#includetemplate) the [`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) that's provided as a part of your GitLab installation. For older versions of GitLab from 11.9 to 12.7, you must -[include](../../../ci/yaml/README.md#includetemplate) the +[include](../../../ci/yaml/index.md#includetemplate) the [`License-Management.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/d2cc841c55d65bc8134bfb3a467e66c36ac32b0a/lib/gitlab/ci/templates/Security/License-Management.gitlab-ci.yml). For GitLab versions earlier than 11.9, you can copy and use the job as defined that template. @@ -115,14 +114,14 @@ the `license_management` job, so you must migrate to the `license_scanning` job `License-Scanning.gitlab-ci.yml` template. The results are saved as a -[License Compliance report artifact](../../../ci/yaml/README.md#artifactsreportslicense_scanning) +[License Compliance report artifact](../../../ci/yaml/index.md#artifactsreportslicense_scanning) that you can later download and analyze. Due to implementation limitations, we always take the latest License Compliance artifact available. Behind the scenes, the [GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/license-finder) is used to detect the languages/frameworks and in turn analyzes the licenses. The License Compliance settings can be changed through [CI/CD variables](#available-cicd-variables) by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. ### When License Compliance runs @@ -180,8 +179,8 @@ directory of your project. ### Overriding the template WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/index.md#rules) instead. If you want to override the job definition (for example, change properties like `variables` or `dependencies`), you need to declare a `license_scanning` job @@ -370,7 +369,7 @@ source "https://gems.example.com" You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by specifying a [`BUNDLE_SSL_CA_CERT`](https://bundler.io/v2.0/man/bundle-config.1.html) -[variable](../../../ci/variables/README.md#custom-cicd-variables) +[variable](../../../ci/variables/index.md#custom-cicd-variables) in the job definition. ### Configuring Cargo projects @@ -394,7 +393,7 @@ To supply a custom root certificate to complete TLS verification, do one of the - Use the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables). - Specify a [`CARGO_HTTP_CAINFO`](https://doc.rust-lang.org/cargo/reference/environment-variables.html) - [variable](../../../ci/variables/README.md#custom-cicd-variables) + [variable](../../../ci/variables/index.md#custom-cicd-variables) in the job definition. ### Configuring Composer projects @@ -427,7 +426,7 @@ For example: You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [CI/CD variable](#available-cicd-variables), or by specifying a [`COMPOSER_CAFILE`](https://getcomposer.org/doc/03-cli.md#composer-cafile) -[variable](../../../ci/variables/README.md#custom-cicd-variables) +[variable](../../../ci/variables/index.md#custom-cicd-variables) in the job definition. ### Configuring Conan projects @@ -441,7 +440,7 @@ documentation for a list of settings that you can apply. The `license_scanning` job runs in a [Debian 10](https://www.debian.org/releases/buster/) Docker image. The supplied image ships with some build tools such as [CMake](https://cmake.org/) and [GCC](https://gcc.gnu.org/). However, not all project types are supported by default. To install additional tools needed to -compile dependencies, use a [`before_script`](../../../ci/yaml/README.md#before_script) +compile dependencies, use a [`before_script`](../../../ci/yaml/index.md#before_script) to install the necessary build tools using the [`apt`](https://wiki.debian.org/PackageManagementTools) package manager. For a comprehensive list, consult [the Conan documentation](https://docs.conan.io/en/latest/introduction.html#all-platforms-all-build-systems-and-compilers). @@ -490,7 +489,7 @@ example: } ``` -If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/README.md#protect-a-cicd-variable) +If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/index.md#protect-a-cicd-variable) following the naming convention described in the [`CONAN_LOGIN_USERNAME` documentation](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name). #### Custom root certificates for Conan @@ -722,7 +721,7 @@ which enables a designated approver that can approve and then merge a merge requ The **Policies** tab in the project's license compliance section displays your project's license policies. Project maintainers can specify policies in this section. - +  Developers of the project can view the policies configured in a project. @@ -857,4 +856,4 @@ root@6abb70e9f193:~# ``` NOTE: -Selecting a custom version of [Mono](https://www.mono-project.com/) or [.NET Core](https://dotnet.microsoft.com/download/dotnet-core) is currently not supported. +Selecting a custom version of [Mono](https://www.mono-project.com/) or [.NET Core](https://dotnet.microsoft.com/download/dotnet) is currently not supported. diff --git a/doc/user/discussions/img/automatically_resolve_outdated_discussions.png b/doc/user/discussions/img/automatically_resolve_outdated_discussions.png Binary files differdeleted file mode 100644 index a6fc4b0aef1..00000000000 --- a/doc/user/discussions/img/automatically_resolve_outdated_discussions.png +++ /dev/null diff --git a/doc/user/discussions/img/btn_new_issue_for_all_threads.png b/doc/user/discussions/img/btn_new_issue_for_all_threads.png Binary files differindex 5d86e0ca016..b07267a011a 100644 --- a/doc/user/discussions/img/btn_new_issue_for_all_threads.png +++ b/doc/user/discussions/img/btn_new_issue_for_all_threads.png diff --git a/doc/user/discussions/img/comment_type_toggle.gif b/doc/user/discussions/img/comment_type_toggle.gif Binary files differdeleted file mode 100644 index b73c197b97f..00000000000 --- a/doc/user/discussions/img/comment_type_toggle.gif +++ /dev/null diff --git a/doc/user/discussions/img/commit_comment_mr_context.png b/doc/user/discussions/img/commit_comment_mr_context.png Binary files differdeleted file mode 100644 index 68f58a57757..00000000000 --- a/doc/user/discussions/img/commit_comment_mr_context.png +++ /dev/null diff --git a/doc/user/discussions/img/commit_comment_mr_discussions_tab.png b/doc/user/discussions/img/commit_comment_mr_discussions_tab.png Binary files differdeleted file mode 100644 index d88f08eae26..00000000000 --- a/doc/user/discussions/img/commit_comment_mr_discussions_tab.png +++ /dev/null diff --git a/doc/user/discussions/img/discussion_lock_system_notes.png b/doc/user/discussions/img/discussion_lock_system_notes.png Binary files differdeleted file mode 100644 index 44a47e3f097..00000000000 --- a/doc/user/discussions/img/discussion_lock_system_notes.png +++ /dev/null diff --git a/doc/user/discussions/img/image_resolved_discussion.png b/doc/user/discussions/img/image_resolved_discussion.png Binary files differdeleted file mode 100644 index f6e5a3b66ae..00000000000 --- a/doc/user/discussions/img/image_resolved_discussion.png +++ /dev/null diff --git a/doc/user/discussions/img/lock_form_member.png b/doc/user/discussions/img/lock_form_member.png Binary files differdeleted file mode 100644 index 7bfcb4faae6..00000000000 --- a/doc/user/discussions/img/lock_form_member.png +++ /dev/null diff --git a/doc/user/discussions/img/lock_form_non_member.png b/doc/user/discussions/img/lock_form_non_member.png Binary files differdeleted file mode 100644 index 59e5fd89499..00000000000 --- a/doc/user/discussions/img/lock_form_non_member.png +++ /dev/null diff --git a/doc/user/discussions/img/merge_request_commits_tab.png b/doc/user/discussions/img/merge_request_commits_tab.png Binary files differdeleted file mode 100644 index 267f3a720dc..00000000000 --- a/doc/user/discussions/img/merge_request_commits_tab.png +++ /dev/null diff --git a/doc/user/discussions/img/new_issue_for_thread.png b/doc/user/discussions/img/new_issue_for_thread.png Binary files differindex 28b76adf7fe..c7f0fd76844 100644 --- a/doc/user/discussions/img/new_issue_for_thread.png +++ b/doc/user/discussions/img/new_issue_for_thread.png diff --git a/doc/user/discussions/img/onion_skin_view.png b/doc/user/discussions/img/onion_skin_view.png Binary files differdeleted file mode 100644 index 81bb4a2c85a..00000000000 --- a/doc/user/discussions/img/onion_skin_view.png +++ /dev/null diff --git a/doc/user/discussions/img/preview_issue_for_thread.png b/doc/user/discussions/img/preview_issue_for_thread.png Binary files differdeleted file mode 100644 index a9d7ab598be..00000000000 --- a/doc/user/discussions/img/preview_issue_for_thread.png +++ /dev/null diff --git a/doc/user/discussions/img/preview_issue_for_threads.png b/doc/user/discussions/img/preview_issue_for_threads.png Binary files differdeleted file mode 100644 index 8757decb178..00000000000 --- a/doc/user/discussions/img/preview_issue_for_threads.png +++ /dev/null diff --git a/doc/user/discussions/img/quickly_assign_commenter_v13_1.png b/doc/user/discussions/img/quickly_assign_commenter_v13_1.png Binary files differindex 7f8ce62fe88..aa8f65ef6c4 100644 --- a/doc/user/discussions/img/quickly_assign_commenter_v13_1.png +++ b/doc/user/discussions/img/quickly_assign_commenter_v13_1.png diff --git a/doc/user/discussions/img/reply_to_comment.gif b/doc/user/discussions/img/reply_to_comment.gif Binary files differdeleted file mode 100644 index c62f7fdb5fe..00000000000 --- a/doc/user/discussions/img/reply_to_comment.gif +++ /dev/null diff --git a/doc/user/discussions/img/resolve_comment_button.png b/doc/user/discussions/img/resolve_comment_button.png Binary files differdeleted file mode 100644 index 0a3ed03a69c..00000000000 --- a/doc/user/discussions/img/resolve_comment_button.png +++ /dev/null diff --git a/doc/user/discussions/img/resolve_thread_button_v13_3.png b/doc/user/discussions/img/resolve_thread_button_v13_3.png Binary files differdeleted file mode 100644 index 1d7b10ce25d..00000000000 --- a/doc/user/discussions/img/resolve_thread_button_v13_3.png +++ /dev/null diff --git a/doc/user/discussions/img/resolve_thread_issue_notice.png b/doc/user/discussions/img/resolve_thread_issue_notice.png Binary files differdeleted file mode 100644 index 30a65b8fbd4..00000000000 --- a/doc/user/discussions/img/resolve_thread_issue_notice.png +++ /dev/null diff --git a/doc/user/discussions/img/resolve_thread_open_issue_v13_9.png b/doc/user/discussions/img/resolve_thread_open_issue_v13_9.png Binary files differdeleted file mode 100644 index 8ff0f5e84dd..00000000000 --- a/doc/user/discussions/img/resolve_thread_open_issue_v13_9.png +++ /dev/null diff --git a/doc/user/discussions/img/start_image_discussion.gif b/doc/user/discussions/img/start_image_discussion.gif Binary files differindex 43efbf2fbb2..18b2a4701cc 100644 --- a/doc/user/discussions/img/start_image_discussion.gif +++ b/doc/user/discussions/img/start_image_discussion.gif diff --git a/doc/user/discussions/img/swipe_view.png b/doc/user/discussions/img/swipe_view.png Binary files differdeleted file mode 100644 index e6f5e5053af..00000000000 --- a/doc/user/discussions/img/swipe_view.png +++ /dev/null diff --git a/doc/user/discussions/img/thread_view.png b/doc/user/discussions/img/thread_view.png Binary files differdeleted file mode 100644 index e2db44aa604..00000000000 --- a/doc/user/discussions/img/thread_view.png +++ /dev/null diff --git a/doc/user/discussions/img/turn_off_lock.png b/doc/user/discussions/img/turn_off_lock.png Binary files differdeleted file mode 100644 index aae1def6f72..00000000000 --- a/doc/user/discussions/img/turn_off_lock.png +++ /dev/null diff --git a/doc/user/discussions/img/turn_on_lock.png b/doc/user/discussions/img/turn_on_lock.png Binary files differdeleted file mode 100644 index f36ffc8831b..00000000000 --- a/doc/user/discussions/img/turn_on_lock.png +++ /dev/null diff --git a/doc/user/discussions/img/two_up_view.png b/doc/user/discussions/img/two_up_view.png Binary files differdeleted file mode 100644 index 3b6ddfbe1be..00000000000 --- a/doc/user/discussions/img/two_up_view.png +++ /dev/null diff --git a/doc/user/discussions/img/unresolved_threads_v14_1.png b/doc/user/discussions/img/unresolved_threads_v14_1.png Binary files differnew file mode 100644 index 00000000000..806dfdc995c --- /dev/null +++ b/doc/user/discussions/img/unresolved_threads_v14_1.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index cf57afb8324..825f9be6ba6 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -5,343 +5,294 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, howto --- -# Threads **(FREE)** +# Comments and threads **(FREE)** GitLab encourages communication through comments, threads, and [code suggestions](../project/merge_requests/reviews/suggestions.md). -For example, you can create a comment in the following places: +There are two types of comments: -- Issues -- Epics -- Merge requests -- Snippets -- Commits -- Commit diffs +- A standard comment. +- A comment in a thread, which has to be resolved. -There are standard comments, and you also have the option to create a comment -in the form of a thread. A comment can also be [turned into a thread](#start-a-thread-by-replying-to-a-standard-comment) -when it receives a reply. +In a comment, you can enter [Markdown](../markdown.md) and use [quick actions](../project/quick_actions.md). -The comment area supports [Markdown](../markdown.md) and [quick actions](../project/quick_actions.md). -You can [suggest code changes](../project/merge_requests/reviews/suggestions.md) in your comment, -which the user can accept through the user interface. You can edit your own -comment at any time, and anyone with the [Maintainer role](../permissions.md) or -higher can also edit a comment made by someone else. - -You can also reply to a comment notification email to reply to the comment if -[Reply by email](../../administration/reply_by_email.md) is configured for your GitLab instance. Replying to a standard comment -creates another standard comment. Replying to a threaded comment creates a reply in the thread. Email replies support -[Markdown](../markdown.md) and [quick actions](../project/quick_actions.md), just as if you replied from the web. +You can [suggest code changes](../project/merge_requests/reviews/suggestions.md) in your commit diff comment, +which the user can accept through the user interface. -NOTE: -There is a limit of 5,000 comments for every object, for example: issue, epic, and merge request. +## Places you can add comments -## Resolvable comments and threads +You can create comments in places like: -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5022) in GitLab 8.11. -> - Resolvable threads can be added only to merge request diffs. -> - Resolving comments individually was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/28750) in GitLab 13.6. - -Thread resolution helps keep track of progress during planning or code review. - -Every thread in merge requests, commits, commit diffs, and -snippets is initially displayed as unresolved. They can then be individually resolved by anyone -with at least Developer access to the project or by the author of the change being reviewed. -If the thread has been resolved and a non-member un-resolves their own response, -this also unresolves the discussion thread. -If the non-member then resolves this same response, this resolves the discussion thread. - -The need to resolve threads prevents you from forgetting to address feedback and lets you -hide threads that are no longer relevant. - - - -### Commit threads in the context of a merge request - -For reviewers with commit-based workflow, it may be useful to add threads to -specific commit diffs in the context of a merge request. These threads -persist through a commit ID change when: - -- force-pushing after a rebase -- amending a commit +- Commit diffs +- Commits +- Designs +- Epics +- Issues +- Merge requests +- Snippets -To create a commit diff thread: +Each object can have as many as 5,000 comments. -1. Navigate to the merge request **Commits** tab. A list of commits that - constitute the merge request are shown. +## Add a comment to a merge request diff -  +You can add comments to a merge request diff. These comments +persist, even when you: -1. Navigate to a specific commit, select the **Changes** tab (where you - are only be presented diffs from the selected commit), and leave a comment. +- Force-push after a rebase. +- Amend a commit. -  +To add a commit diff comment: -1. Any threads created this way are shown in the merge request's - **Discussions** tab and are resolvable. +1. To select a specific commit, on the merge request, select the **Commits** tab, select the commit + message. To view the latest commit, select the **Changes** tab. +1. By the line you want to comment on, hover over the line number and select **{comment}**. + You can select multiple lines by dragging the **{comment}** icon. +1. Type your comment and select **Start a review** or **Add comment now**. -  +The comment is displayed on the merge request's **Discussions** tab. -Threads created this way only appear in the original merge request -and not when navigating to that commit under your project's -**Repository > Commits** page. +The comment is not displayed on your project's **Repository > Commits** page. NOTE: -When a link of a commit reference is found in a thread inside a merge -request, it is automatically converted to a link in the context of the -current merge request. - -### Marking a comment or thread as resolved +When your comment contains a reference to a commit included in the merge request, +it's automatically converted to a link in the context of the current merge request. +For example, `28719b171a056960dfdc0012b625d0b47b123196` becomes +`https://gitlab.example.com/example-group/example-project/-/merge_requests/12345/diffs?commit_id=28719b171a056960dfdc0012b625d0b47b123196`. -You can mark a thread as resolved by selecting the **Resolve thread** -button at the bottom of the thread. +## Add a comment to a commit - +You can add comments and threads to a particular commit. -Alternatively, you can mark each comment as resolved individually. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Commits**. +1. Below the commits, in the **Comment** field, enter a comment. +1. Select **Comment** or select the down arrow (**{chevron-down}**) to select **Start thread**. - - -### Move all unresolved threads in a merge request to an issue +WARNING: +Threads created this way are lost if the commit ID changes after a +force push. -To continue all open threads from a merge request in a new issue, select -**Resolve all threads in new issue**. +## Add a comment to an image - +In merge requests and commit detail views, you can add a comment to an image. +This comment can also be a thread. -Alternatively, when your project only accepts merge requests [when all threads -are resolved](#only-allow-merge-requests-to-be-merged-if-all-threads-are-resolved), -an **open an issue to resolve them later** link displays in the merge -request widget. +1. Hover your mouse over the image. +1. Select the location where you want to comment. - +An icon is displayed on the image and a comment field is displayed. -This prepares an issue with its content referring to the merge request and -the unresolved threads. + - +## Reply to a comment by sending email -Hitting **Create issue** causes all threads to be marked as resolved and -add a note referring to the newly created issue. +If you have ["reply by email"](../../administration/reply_by_email.md) configured, +you can reply to comments by sending an email. - +- When you reply to a standard comment, another standard comment is created. +- When you reply to a threaded comment, it creates a reply in the thread. -You can now proceed to merge the merge request from the UI. +You can use [Markdown](../markdown.md) and [quick actions](../project/quick_actions.md) in your email replies. -### Moving a single thread to a new issue +## Who can edit comments -To create a new issue for a single thread, you can use the **Resolve this -thread in a new issue** button. +You can edit your own comment at any time. - +Anyone with the [Maintainer role](../permissions.md) or +higher can also edit a comment made by someone else. -This directs you to a new issue prefilled with the content of the -thread, similar to the issues created for delegating multiple -threads at once. Saving the issue marks the thread as resolved and -add a note to the merge request thread referencing the new issue. +## Prevent comments by locking an issue - +You can prevent public comments in an issue or merge request. +When you do, only project members can add and edit comments. -### Only allow merge requests to be merged if all threads are resolved +Prerequisite: -You can prevent merge requests from being merged until all threads are -resolved. +- In merge requests, you must have at least the Developer role. +- In issues, you must have at least the Reporter role. -Navigate to your project's settings page, select the -**Only allow merge requests to be merged if all threads are resolved** check -box and hit **Save** for the changes to take effect. +1. On the right sidebar, next to **Lock issue** or **Lock merge request**, select **Edit**. +1. On the confirmation dialog, select **Lock**. - +Notes are added to the page details. -From now on, you can't merge from the UI until all threads -are resolved. +If an issue or merge request is locked and closed, you cannot reopen it. - +## Mark a comment as confidential -### Automatically resolve merge request diff threads when they become outdated +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207473) in GitLab 13.9. +> - [Deployed behind a feature flag](../feature_flags.md), disabled by default. +> - Disabled on GitLab.com. +> - Not recommended for production use. +> - To use in GitLab self-managed instances, ask a GitLab administrator to enable it. **(FREE SELF)** -You can automatically resolve merge request diff threads on lines modified -with a new push. +WARNING: +This feature might not be available to you. Check the **version history** note above for details. -Navigate to your project's settings page, select the **Automatically resolve -merge request diffs threads on lines changed with a push** check box and hit -**Save** for the changes to take effect. +You can make a comment confidential, so that it is visible only to project members +who have at least the Reporter role. - +1. Below the comment, select the **Make this comment confidential** checkbox. +1. Select **Comment**. -From now on, any threads on a diff are resolved by default if a push -makes that diff section outdated. Threads on lines that don't change and -top-level resolvable threads are not automatically resolved. + -## Commit threads +## Show only comments -You can add comments and threads to a particular commit under your -project's **Repository > Commits**. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/26723) in GitLab 11.5. -WARNING: -Threads created this way are lost if the commit ID changes after a -force push. +For issues and merge requests with many comments, you can filter the page to show comments only. -## Threaded discussions +1. Open a merge request's **Discussion** tab, or epic or issue's **Overview** tab. +1. On the right side of the page, select from the filter: + - **Show all activity**: Display all user comments and system notes + (issue updates, mentions from other issues, changes to the description, and so on). + - **Show comments only**: Display only user comments. + - **Show history only**: Display only activity notes. -While resolvable threads are only available to merge request diffs, -threads can also be added without a diff. You can start a specific -thread which looks like a thread, on issues, commits, snippets, and -merge requests. + -To start a threaded discussion, select the **Comment** button toggle dropdown, -select **Start thread**, and then select **Start thread** when you're ready to -post the comment. +GitLab saves your preference, so it persists when you visit the same page again +from any device you're logged into. - +## Assign an issue to the commenting user -This posts a comment with a single thread to allow you to discuss specific -comments in greater detail. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/191455) in GitLab 13.1. - +You can assign an issue to a user who made a comment. -## Image threads +1. In the comment, select the **More Actions** menu. +1. Select **Assign to commenting user**. -Sometimes a thread is revolved around an image. With image threads, -you can easily target a specific coordinate of an image and start a thread -around it. Image threads are available in merge requests and commit detail views. + -To start an image thread, hover your mouse over the image. Your mouse pointer -should convert into an icon, indicating that the image is available for commenting. -Simply click anywhere on the image to create a new thread. +Select the button again to unassign the commenter. - +## Create a thread by replying to a standard comment -After you select the image, a comment form is displayed that would be the start -of your thread. After you save your comment, a new badge is displayed on -top of your image. This badge represents your thread. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30299) in GitLab 11.9. -NOTE: -This thread badge is typically associated with a number that is only used as a visual -reference for each thread. In the merge request thread tab, -this badge is indicated with a comment icon, because each thread renders a new -image section. +When you reply to a standard comment, you create a thread. -Image threads also work on diffs that replace an existing image. In this diff view -mode, you can toggle the different view modes and still see the thread point badges. +Prerequisites: -| 2-up | Swipe | Onion Skin | -|:-----------:|:----------:|:----------:| -|  |  |  | +- You must have at least the [Guest role](../permissions.md#project-members-permissions). +- You must be in an issue, merge request, or epic. Commits and snippets threads are not supported. -Image threads also work well with resolvable threads. Resolved threads -on diffs (not on the merge request discussion tab) appear collapsed on page -load and have a corresponding badge counter to match the counter on the image. +To create a thread by replying to a comment: - +1. On the top right of the comment, select **{comment}** (**Reply to comment**). -## Lock discussions +  -For large projects with many contributors, it may be useful to stop threads -in issues or merge requests in these scenarios: + The reply area is displayed. -- The project maintainer has already resolved the thread and it is not helpful - for continued feedback. -- The project maintainer has already directed new conversation - to newer issues or merge requests. -- The people participating in the thread are trolling, abusive, or otherwise - being unproductive. +1. Type your reply. +1. Select **Comment** or **Add comment now** (depending on where in the UI you are replying). -In these cases, a user with Developer permissions or higher in the project can lock (and unlock) -an issue or a merge request, using the "Lock" section in the sidebar. For issues, -a user with Reporter permissions can lock (and unlock). +The top comment is converted to a thread. -| Unlock | Lock | -| :-----------: | :----------: | -|  |  | +## Create a thread without replying to a comment -System notes indicate locking and unlocking. +You can create a thread without replying to a standard comment. - +Prerequisites: -In a locked issue or merge request, only team members can add new comments and -edit existing comments. Non-team members are restricted from adding or editing comments. +- You must have at least the [Guest role](../permissions.md#project-members-permissions). +- You must be in an issue, merge request, commit, or snippet. -| Team member | Non-team member | -| :-----------: | :----------: | -|  |  | +To create a thread: -Additionally, locked issues and merge requests can't be reopened. +1. Type a comment. +1. Below the comment, to the right of the **Comment** button, select the down arrow (**{chevron-down}**). +1. From the list, select **Start thread**. +1. Select **Start thread** again. -## Confidential Comments +A threaded comment is created. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207473) in GitLab 13.9. -> - [Deployed behind a feature flag](../feature_flags.md), disabled by default. -> - Disabled on GitLab.com. -> - Not recommended for production use. -> - To use in GitLab self-managed instances, ask a GitLab administrator to enable it. **(FREE SELF)** + -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +## Resolve a thread -When creating a comment, you can make it visible only to the project members (users with Reporter and higher permissions). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5022) in GitLab 8.11. +> - Resolvable threads can be added only to merge request diffs. +> - Resolving comments individually was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/28750) in GitLab 13.6. -To create a confidential comment, select the **Make this comment confidential** check box before you submit it. +You can resolve a thread when you want to finish a conversation. - +Prerequisites: -## Filtering notes +- You must have at least the [Developer role](../permissions.md#project-members-permissions) + or be the author of the change being reviewed. +- You must be in an issue, merge request, commit, or snippet. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/26723) in GitLab 11.5. +To resolve a thread: -For issues with many comments like activity notes and user comments, sometimes -finding useful information can be hard. There is a way to filter comments from single notes and threads for merge requests and issues. +1. Go to the thread. +1. Do one of the following: + - In the top right of the original comment, select the **Resolve thread** (**{check-circle}**) icon. + - Below the last reply, in the **Reply** field, select **Resolve thread**. + - Below the last reply, in the **Reply** field, enter text, select the **Resolve thread** checkbox, and select **Add comment now**. -From a merge request's **Discussion** tab, or from an epic/issue overview, find the filter's dropdown menu on the right side of the page, from which you can choose one of the following options: +At the top of the page, the number of unresolved threads is updated. -- **Show all activity**: displays all user comments and system notes - (issue updates, mentions from other issues, changes to the description, etc). -- **Show comments only**: only displays user comments in the list. -- **Show history only**: only displays activity notes. + - +### Move all unresolved threads in a merge request to an issue -After you select one of the filters in a given issue or merge request, GitLab saves -your preference, so that it persists when you visit the same page again -from any device you're logged into. +If you have multiple unresolved threads in a merge request, you can +create an issue to resolve them separately. -## Start a thread by replying to a standard comment +- In the merge request, at the top of the page, select **Resolve all threads in new issue**. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30299) in GitLab 11.9 +  -To reply to a standard (non-thread) comment, you can use the **Reply to comment** button. +All threads are marked as resolved and a link is added from the merge request to +the newly created issue. - +### Move one unresolved thread in a merge request to an issue -The **Reply to comment** button is only displayed if you have permissions to reply to an existing thread, or start a thread from a standard comment. +If you have one specific unresolved thread in a merge request, you can +create an issue to resolve it separately. -Selecting the **Reply to comment** button brings the reply area into focus and you can type your reply. +- In the merge request, under the last reply to the thread, next to the + **Resolve thread** button, select **Resolve this thread in a new issue**. - +  -Replying to a non-thread comment converts the non-thread comment to a -thread after the reply is submitted. This conversion is considered an edit -to the original comment, so a note about when it was last edited appears underneath it. +The thread is marked as resolved and a link is added from the merge request to +the newly created issue. -This feature exists only for issues, merge requests, and epics. Commits, snippets, and merge request diff threads are -not supported yet. +### Prevent merge unless all threads are resolved -## Assign an issue to the commenting user +You can prevent merge requests from being merged until all threads are +resolved. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/191455) in GitLab 13.1. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. Under **Merge checks**, select the **All discussions must be resolved** checkbox. +1. Select **Save changes**. -You can assign an issue to a user who made a comment. +### Automatically resolve threads in a merge request when they become outdated -In the comment, select the **More Actions** menu, and then select **Assign to commenting user**. +You can set merge requests to automatically resolve threads when lines are modified +with a new push. -Select the button again to unassign the commenter. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. Under **Merge options**, select the + **Automatically resolve merge request diff discussions when they become outdated** checkbox. +1. Select **Save changes**. - +Threads are now resolved if a push makes a diff section outdated. +Threads on lines that don't change and top-level resolvable threads are not resolved. -## Enable or disable Confidential Comments **(FREE SELF)** +## Enable or disable confidential comments **(FREE SELF)** -Confidential Comments is under development and not ready for production use. It is +Confidential comments are under development and not ready for production use. The feature is deployed behind a feature flag that is **disabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) can enable it. diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index f371de30b88..ef458db67f0 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -6,12 +6,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab.com settings **(FREE SAAS)** -This page contains information about the settings that are used on -[GitLab.com](https://about.gitlab.com/pricing/). +This page contains information about the settings that are used on GitLab.com, available to +[GitLab SaaS](https://about.gitlab.com/pricing/) customers. ## SSH host keys fingerprints -Below are the fingerprints for GitLab.com's SSH host keys. The first time you +Below are the fingerprints for SSH host keys on GitLab.com. The first time you connect to a GitLab.com repository, one of these keys is displayed in the output. | Algorithm | MD5 (deprecated) | SHA256 | @@ -39,6 +39,13 @@ and has its own dedicated IP address (`192.237.158.143`). The IP address for `mg.gitlab.com` is subject to change at any time. +### Service Desk custom mailbox + +On GitLab.com, there's a mailbox configured for Service Desk with the email address: +`contact-project+%{key}@incoming.gitlab.com`. To use this mailbox, configure the +[custom suffix](../project/service_desk.md#configuring-a-custom-email-address-suffix) in project +settings. + ## Backups [See our backup strategy](https://about.gitlab.com/handbook/engineering/infrastructure/production/#backups). @@ -97,14 +104,14 @@ which is part of [GitLab CI/CD](#gitlab-cicd). ## GitLab CI/CD -Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). +Below are the current settings regarding [GitLab CI/CD](../../ci/index.md). Any settings or feature limits not listed here are using the defaults listed in the related documentation. -| Setting | GitLab.com | Default | -|-------------------------------------|------------|---------| -| Artifacts maximum size (compressed) | 1 GB | 100 MB | -| Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | +| Setting | GitLab.com | Default | +|-------------------------------------|-------------|---------| +| Artifacts maximum size (compressed) | 1 GB | 100 MB | +| Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | | Scheduled Pipeline Cron | `*/5 * * * *` | `3-59/10 * * * *` | | [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited | | [Max CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) | `2` | Unlimited | @@ -118,14 +125,14 @@ the related documentation. GitLab.com has the following [account limits](../admin_area/settings/account_and_limit_settings.md) enabled. If a setting is not listed, it is set to the default value. -If you are near or over the repository size limit, you can -[reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). +If you are near or over the repository size limit, you can either +[reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md) or [purchase additional storage](https://about.gitlab.com/pricing/licensing-faq/#can-i-buy-more-storage). -| Setting | GitLab.com | Default | -|-------------------------------|------------|---------| +| Setting | GitLab.com | Default | +|-------------------------------|-------------|---------| | [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | Unlimited | -| Maximum import size | 5 GB | Unlimited ([Modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to unlimited in GitLab 13.8. | -| Maximum attachment size | 10 MB | 10 MB | +| Maximum import size | 5 GB | Unlimited ([Modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to unlimited in GitLab 13.8.) | +| Maximum attachment size | 10 MB | 10 MB | NOTE: `git push` and GitLab project imports are limited to 5 GB per request through @@ -138,7 +145,7 @@ GitLab.com uses the IP ranges `34.74.90.64/28` and `34.74.226.0/24` for traffic fleet. This whole range is solely allocated to GitLab. You can expect connections from webhooks or repository mirroring to come from those IPs and allow them. -GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)). +GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com, you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)). For outgoing connections from CI/CD runners, we are not providing static IP addresses. All GitLab runners are deployed into Google Cloud Platform (GCP). Any @@ -164,32 +171,32 @@ also load certain page content directly from common public CDN hostnames. The following limits apply for [Webhooks](../project/integrations/webhooks.md): -| Setting | GitLab.com | Default | -|----------------------|------------|---------| +| Setting | GitLab.com | Default | +|----------------------|-------------|---------| | [Webhook rate limit](../../administration/instance_limits.md#webhook-rate-limit) | `120` calls per minute for GitLab Free, unlimited for GitLab Premium and GitLab Ultimate | Unlimited | | [Number of webhooks](../../administration/instance_limits.md#number-of-webhooks) | `100` per project, `50` per group | `100` per project, `50` per group | -| Maximum payload size | 25 MB | 25 MB | +| Maximum payload size | 25 MB | 25 MB | ## Shared runners GitLab has shared runners on GitLab.com that you can use to run your CI jobs. -For more information, see [choosing a runner](../../ci/runners/README.md). +For more information, see [choosing a runner](../../ci/runners/index.md). ## Sidekiq GitLab.com runs [Sidekiq](https://sidekiq.org) with arguments `--timeout=4 --concurrency=4` and the following environment variables: -| Setting | GitLab.com | Default | -|----------------------------------------|------------|-----------| -| `SIDEKIQ_DAEMON_MEMORY_KILLER` | - | `1` | -| `SIDEKIQ_MEMORY_KILLER_MAX_RSS` | `2000000` | `2000000` | -| `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS` | - | - | -| `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL` | - | `3` | -| `SIDEKIQ_MEMORY_KILLER_GRACE_TIME` | - | `900` | -| `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT` | - | `30` | -| `SIDEKIQ_LOG_ARGUMENTS` | `1` | `1` | +| Setting | GitLab.com | Default | +|----------------------------------------|-------------|-----------| +| `SIDEKIQ_DAEMON_MEMORY_KILLER` | - | `1` | +| `SIDEKIQ_MEMORY_KILLER_MAX_RSS` | `2000000` | `2000000` | +| `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS` | - | - | +| `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL` | - | `3` | +| `SIDEKIQ_MEMORY_KILLER_GRACE_TIME` | - | `900` | +| `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT` | - | `30` | +| `SIDEKIQ_LOG_ARGUMENTS` | `1` | `1` | NOTE: The `SIDEKIQ_MEMORY_KILLER_MAX_RSS` setting is `16000000` on Sidekiq import @@ -362,7 +369,7 @@ See [non-configurable limits](../../security/rate_limits.md#non-configurable-lim for information on rate limits that are not configurable, and therefore also used on GitLab.com. -## GitLab.com Logging +## GitLab.com logging We use [Fluentd](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#fluentd) to parse our logs. Fluentd sends our logs to @@ -377,7 +384,7 @@ You can view more information in our runbooks such as: - Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#retention) - A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#logging-infrastructure-overview) -### Job Logs +### Job logs By default, GitLab does not expire job logs. Job logs are retained indefinitely, and can't be configured on GitLab.com to expire. You can erase job logs @@ -390,7 +397,7 @@ In addition to the GitLab Enterprise Edition Omnibus install, GitLab.com uses the following applications and settings to achieve scale. All settings are publicly available at [chef cookbooks](https://gitlab.com/gitlab-cookbooks). -### Elastic Cluster +### Elastic cluster We use Elasticsearch and Kibana for part of our monitoring solution: diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 4de464822f7..0d885183a41 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -66,7 +66,7 @@ automatically. If you're using [Auto DevOps](../../../topics/autodevops/index.md for deployments with a cluster not managed by GitLab, you must ensure: - The project's deployment service account has permissions to deploy to - [`KUBE_NAMESPACE`](../../project/clusters/index.md#deployment-variables). + [`KUBE_NAMESPACE`](../../project/clusters/deploy_to_cluster.md#deployment-variables). - `KUBECONFIG` correctly reflects any changes to `KUBE_NAMESPACE` (this is [not automatic](https://gitlab.com/gitlab-org/gitlab/-/issues/31519)). Editing `KUBE_NAMESPACE` directly is discouraged. @@ -96,14 +96,14 @@ per [multiple Kubernetes clusters](#multiple-kubernetes-clusters) When specifyin this is automatically set as an environment variable (`KUBE_INGRESS_BASE_DOMAIN`) during the [Auto DevOps](../../../topics/autodevops/index.md) stages. -The domain should have a wildcard DNS configured to the Ingress IP address. [More details](../../project/clusters/index.md#base-domain). +The domain should have a wildcard DNS configured to the Ingress IP address. [More details](../../project/clusters/gitlab_managed_clusters.md#base-domain). ## Environment scopes **(PREMIUM)** When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the -[environment-specific CI/CD variables](../../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable) +[environment-specific CI/CD variables](../../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable) work. While evaluating which environment matches the environment scope of a @@ -122,7 +122,7 @@ For example, if your project has the following Kubernetes clusters: | Test | `test` | Group | | Development| `*` | Group | -And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/README.md): +And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/index.md): ```yaml stages: @@ -163,7 +163,7 @@ are deployed to the Kubernetes cluster, see the documentation for ## Security of runners For important information about securely configuring runners, see -[Security of runners](../../project/clusters/add_remove_clusters.md#security-of-runners) +[Security of runners](../../project/clusters/cluster_access.md#security-of-runners) documentation for project-level clusters. ## More information diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index 12d7eaf2f12..670ecc48370 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -8,12 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3090) in GitLab 12.2 for subgroups. -With Contribution Analytics you can get an overview of the following activity in your -group: - -- Issues -- Merge requests -- Push events +With Contribution Analytics you can get an overview of the [contribution actions](../../../api/events.md#action-types) in your +group. To view the Contribution Analytics, go to your group and select **Analytics > Contribution**. diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md index d544003536e..41046477b90 100644 --- a/doc/user/group/custom_project_templates.md +++ b/doc/user/group/custom_project_templates.md @@ -9,25 +9,35 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6861) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.6. -Custom project templates are useful for organizations that need to create many similar types of -[projects](../project/index.md). -Projects created from these templates serve as a common starting point. +[Group owners](../permissions.md#group-members-permissions) can set a subgroup to +be the source of project templates that are selectable when a new project is created +in the group. These templates can be selected when you go to **New project > Create from template** +in the group and select the **Group** tab. -## Setting up group-level project templates +Every project in the subgroup, but not nested subgroups, can be selected by members +of the group when a new project is created. -To use a custom project template for a new project: +Repository and database information that is copied over to each new project is identical to the +data exported with the [GitLab Project Import/Export](../project/settings/import_export.md). + +To set custom project templates at the instance level, see [Custom instance-level project templates](../admin_area/custom_project_templates.md). -1. [Create a `templates` subgroup](subgroups/index.md). -1. [Add repositories (projects) to that new subgroup](index.md#add-projects-to-a-group), - as your templates. -1. Edit your group's settings to look to your _templates_ subgroup for templates: +## Set up group-level project templates - 1. In the left menu, select **Settings > General**. If you don't have access to the - group's settings, you may not have sufficient privileges (for example, you may need developer - or higher permissions). - 1. Scroll to **Custom project templates** and select **Expand**. If no **Custom project templates** - section displays, make sure you've created a subgroup and added a project (repository) to it. - 1. Select the **templates** subgroup. +To set up custom project templates in a group, add the subgroup that contains the +project templates to the group settings: + +1. In the group, create a [subgroup](subgroups/index.md). +1. [Add projects to the new subgroup](index.md#add-projects-to-a-group) as your templates. +1. In the left menu for the group, go to **Settings > General**. +1. Expand **Custom project templates** and select the subgroup. + +If all enabled [project features](../project/settings/index.md#sharing-and-permissions) +(except for GitLab Pages) are set to **Everyone With Access**, then every project +template in the subgroup is available to every member of the group. + +Any projects added to the subgroup later can be selected the next time a group member +creates a new project. ### Example structure @@ -54,25 +64,6 @@ gitlab.com/acmeco/ ... ``` -### Adjust Settings - -Users can configure a GitLab group that serves as template source under a group's -**Settings > General > Custom project templates**. - -NOTE: -GitLab administrators can [set project templates for an entire GitLab instance](../admin_area/custom_project_templates.md). - -Within this section, you can configure the group where all the custom project templates are sourced. -If all enabled [project features](../project/settings/index.md#sharing-and-permissions) -(except for GitLab Pages) are set to **Everyone With Access**, then every project template directly -under the group namespace is available to every signed-in user. However, private projects are -available only if the user is a member of the project. Also note that only direct subgroups can be -set as the template source. Projects of nested subgroups of a selected template source cannot be -used. - -Repository and database information that are copied over to each new project are identical to the -data exported with the [GitLab Project Import/Export](../project/settings/import_export.md). - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/group/devops_adoption/img/group_devops_adoption_v14_0.png b/doc/user/group/devops_adoption/img/group_devops_adoption_v14_0.png Binary files differdeleted file mode 100644 index 91055f660da..00000000000 --- a/doc/user/group/devops_adoption/img/group_devops_adoption_v14_0.png +++ /dev/null diff --git a/doc/user/group/devops_adoption/img/group_devops_adoption_v14_1.png b/doc/user/group/devops_adoption/img/group_devops_adoption_v14_1.png Binary files differnew file mode 100644 index 00000000000..a790a560a9b --- /dev/null +++ b/doc/user/group/devops_adoption/img/group_devops_adoption_v14_1.png diff --git a/doc/user/group/devops_adoption/index.md b/doc/user/group/devops_adoption/index.md index f98325143cc..4332f261481 100644 --- a/doc/user/group/devops_adoption/index.md +++ b/doc/user/group/devops_adoption/index.md @@ -7,14 +7,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group DevOps Adoption **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321083) in GitLab 13.11 as a [Beta feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta). -> - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/323159) in GitLab 13.12. -> - Enabled on GitLab.com. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-group-devops-adoption). **(ULTIMATE SELF)** - -This in-development feature might not be available for your use. There can be -[risks when enabling features still in development](../../feature_flags.md#risks-when-enabling-features-still-in-development). -Refer to this feature's version history for more details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333556) in GitLab 14.1. +> - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. +> - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. Prerequisites: @@ -25,16 +20,17 @@ To access Group DevOps Adoption, go to your group and select **Analytics > DevOp Group DevOps Adoption shows you how individual groups and sub-groups within your organization use the following features: - Dev - - Issues - - Merge Requests - Approvals - Code owners + - Issues + - Merge requests - Sec - - Scans + - DAST + - SAST - Ops - - Runners - - Pipelines - Deployments + - Pipelines + - Runners When managing groups in the UI, you can add your sub-groups with the **Add sub-group to table** button, in the top right hand section of your Groups pages. @@ -45,7 +41,7 @@ With DevOps Adoption you can: - Identify specific sub-groups that are lagging in their adoption of GitLab so you can help them along in their DevOps journey. - Find the sub-groups that have adopted certain features and can provide guidance to other sub-groups on how to use those features. - + ## Enable data processing @@ -64,12 +60,6 @@ Each group appears as a separate row in the table. For each row, a feature is considered "adopted" if it has been used in a project in the given group during the time period (including projects in any sub-groups of the given group). -You should expect adoption to be lower at the beginning of the month, -before you have had an opportunity to use all the features listed in the table. - -In the future [we plan to implement](https://gitlab.com/gitlab-org/gitlab/-/issues/329708) -a rolling 30-day perspective instead. - ## When is a feature considered adopted A feature is considered "adopted" if it has been used anywhere in the group in the specified time. @@ -100,26 +90,3 @@ To add a sub-group to your Group DevOps Adoption report: The sub-group data might not appear immediately, because GitLab requires around a minute to collect the data. - -Please note that the sub-group data might not appear immediately, -because GitLab requires a few moments to collect the data. -Generally the data will be visible in less than one minute. - -## Enable or disable Group DevOps Adoption **(ULTIMATE SELF)** - -Group DevOps Adoption is under development and not ready for production use. It is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it. - -To disable it: - -```ruby -Feature.disable(:group_devops_adoption) -``` - -To re-enable it: - -```ruby -Feature.enable(:group_devops_adoption) -``` diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md index 3b653a7f3f8..34eebfb9e3b 100644 --- a/doc/user/group/epics/epic_boards.md +++ b/doc/user/group/epics/epic_boards.md @@ -7,17 +7,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Epic Boards **(PREMIUM)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5067) in GitLab 13.10. -> - [Deployed behind a feature flag](../../feature_flags.md), disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/290039) in GitLab 14.0. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](../../../administration/feature_flags.md). +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290039) in GitLab 14.1. Epic boards build on the existing [epic tracking functionality](index.md) and [labels](../../project/labels.md). Your epics appear as cards in vertical lists, organized by their assigned labels. -To view an epic board, in a group, select **Epics > Boards**. +To view an epic board: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Epics > Boards**.  @@ -29,7 +28,8 @@ Prerequisites: To create a new epic board: -1. Go to your group and select **Epics > Boards**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Epics > Boards**. 1. In the upper left corner, select the dropdown with the current board name. 1. Select **Create new board**. 1. Enter the new board's title. @@ -77,7 +77,8 @@ Prerequisites: To create a new list: -1. Go to your group and select **Epics > Boards**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Epics > Boards**. 1. In the upper-right corner, select **Create list**. 1. In the **New list** column expand the **Select a label** dropdown and select the label to use as list scope. @@ -129,6 +130,15 @@ You can filter by the following: - Author - Label +### View count of issues, weight, and progress of an epic + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331330) in GitLab 14.1. + +Epics on an epic board show a summary of their issues, weight, and progress. +To see the number of open and closed issues and the completed and incomplete +weight, hover over the issues icon **{issues}**, weight icon **{weight}**, or +progress icon **{progress}**. + ### Move epics and lists > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5079) in GitLab 14.0. @@ -170,22 +180,3 @@ To edit the scope of an epic board: - Show or hide the Open and Closed columns. - Select other labels as the board's scope. 1. Select **Save changes**. - -## Enable or disable epic boards - -Epic boards are under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it. - -To disable it: - -```ruby -Feature.disable(:epic_boards) -``` - -To enable it: - -```ruby -Feature.enable(:epic_boards) -``` diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 89fd32a8db1..f6c3ea6c090 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -5,8 +5,6 @@ group: Product Planning info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -<!-- When adding a new h2 section here, remember to mention it in index.md#manage-epics --> - # Manage epics **(PREMIUM)** This page collects instructions for all the things you can do with [epics](index.md) or in relation @@ -140,6 +138,7 @@ link in the issue sidebar. > - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.5. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) to the [Premium](https://about.gitlab.com/pricing/) tier in GitLab 12.8. > - Searching by the user's reaction emoji [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325630) in GitLab 13.11. +> - Sorting by epic titles [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331625) in GitLab 14.1. You can search for an epic from the list of epics using filtered search bar (similar to that of issues and merge requests) based on following parameters: @@ -162,6 +161,7 @@ You can also sort epics list by: - Last updated - Start date - Due date +- Title Each option contains a button that can toggle the order between **Ascending** and **Descending**. The sort option and order is saved and used wherever you browse epics, including the diff --git a/doc/user/group/import/img/bulk_imports_v13_8.png b/doc/user/group/import/img/bulk_imports_v13_8.png Binary files differdeleted file mode 100644 index ae4d8567d80..00000000000 --- a/doc/user/group/import/img/bulk_imports_v13_8.png +++ /dev/null diff --git a/doc/user/group/import/img/bulk_imports_v14_1.png b/doc/user/group/import/img/bulk_imports_v14_1.png Binary files differnew file mode 100644 index 00000000000..fb419c1df6c --- /dev/null +++ b/doc/user/group/import/img/bulk_imports_v14_1.png diff --git a/doc/user/group/import/img/import_panel_v13_8.png b/doc/user/group/import/img/import_panel_v13_8.png Binary files differdeleted file mode 100644 index 28d61785098..00000000000 --- a/doc/user/group/import/img/import_panel_v13_8.png +++ /dev/null diff --git a/doc/user/group/import/img/import_panel_v14_1.png b/doc/user/group/import/img/import_panel_v14_1.png Binary files differnew file mode 100644 index 00000000000..28417383b6c --- /dev/null +++ b/doc/user/group/import/img/import_panel_v14_1.png diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index 8bf995a4fd9..d76685f992b 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -22,6 +22,7 @@ The following resources are migrated to the target instance: - description - attributes - subgroups + - avatar ([Introduced in 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/322904)) - Group Labels ([Introduced in 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/292429)) - title - description @@ -109,7 +110,7 @@ on an existing group's page. 1. On the New Group page, select **Import group**. -  +  1. Fill in source URL of your GitLab. 1. Fill in [personal access token](../../../user/profile/personal_access_tokens.md) for remote GitLab instance. @@ -118,14 +119,14 @@ on an existing group's page. ### Selecting which groups to import After you have authorized access to the GitLab instance, you are redirected to the GitLab Group -Migration importer page. Your remote GitLab groups, which you have Owner access to, are listed. +Migration importer page. Listed are the remote GitLab groups to which you have the Owner role. 1. By default, the proposed group namespaces match the names as they exist in remote instance, but based on your permissions, you can choose to edit these names before you proceed to import any of them. 1. Select the **Import** button next to any number of groups. -1. The **Status** column shows the import status of each group. You can choose to leave the page open and it will update in real-time. +1. The **Status** column shows the import status of each group. You can choose to leave the page open and it updates in real-time. 1. Once a group has been imported, click its GitLab path to open its GitLab URL. - + diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 15fbb442752..787461f9d96 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -29,6 +29,21 @@ To view groups: You can also view groups by namespace. +### Group visibility + +Like projects, a group can be configured to limit the visibility of it to: + +- Anonymous users. +- All signed-in users. +- Only explicit group members. + +The restriction for [visibility levels](../admin_area/settings/visibility_and_access_controls.md#restricted-visibility-levels) +on the application setting level also applies to groups. If set to internal, the explore page is +empty for anonymous users. The group page has a visibility level icon. + +Administrator users cannot create a subgroup or project with a higher visibility level than that of +the immediate parent group. + ### Namespaces In GitLab, a namespace is a unique name and URL for a user, a group, or subgroup. diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 38d209f04ca..5a435f32569 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -11,9 +11,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Deployed behind a feature flag, disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/221047) in GitLab 13.2. > - Enabled on GitLab.com. -> - Able to be enabled or disabled per-group. +> - Can be enabled or disabled per-group. > - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-iterations). **(PREMIUM ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-iterations). **(PREMIUM ONLY)** > - Moved to GitLab Premium in 13.9. Iterations are a way to track issues over a period of time. This allows teams @@ -32,31 +32,85 @@ In GitLab, iterations are similar to milestones, with a few differences: - Iterations require both a start and an end date. - Iteration date ranges cannot overlap. +## Iteration cadences + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5077) in GitLab 14.1. +> - Deployed behind a [feature flag](../../feature_flags.md), disabled by default. +> - Disabled on GitLab.com. +> - Not recommended for production use. +> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-iteration-cadences). **(PREMIUM SELF)** + +This in-development feature might not be available for your use. There can be +[risks when enabling features still in development](../../feature_flags.md#risks-when-enabling-features-still-in-development). +Refer to this feature's version history for more details. + +Iteration cadences automate some common iteration tasks. They can be used to +automatically create iterations every 1, 2, 3, 4, or 6 weeks. They can also +be configured to automatically roll over incomplete issues to the next iteration. + +With iteration cadences enabled, you must first +[create an iteration cadence](#create-an-iteration-cadence) before you can +[create an iteration](#create-an-iteration). + +### Create an iteration cadence + +Prerequisites: + +- You must have at least the [Developer role](../../permissions.md) for a group. + +To create an iteration cadence: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. +1. Select **New iteration cadence**. +1. Fill out required fields, and select **Create iteration cadence**. The cadence list page opens. + +### Delete an iteration cadence + +Prerequisites: + +- You must have at least the [Developer role](../../permissions.md) for a group. + +Deleting an iteration cadence also deletes all iterations within that cadence. + +To delete an iteration cadence: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. +1. Select the three-dot menu (**{ellipsis_v}**) > **Delete cadence** for the cadence you want to delete. +1. Select **Delete cadence** in the confirmation modal. + ## View the iterations list -To view the iterations list, in a group, go to **{issues}** **Issues > Iterations**. -From there you can create a new iteration or click an iteration to get a more detailed view. +To view the iterations list, go to **{issues}** **Issues > Iterations**. +To view all the iterations in a cadence, ordered by descending date, select that iteration cadence. +From there you can create a new iteration or select an iteration to get a more detailed view. ## Create an iteration -NOTE: -You need Developer [permissions](../../permissions.md) or higher to create an iteration. +Prerequisites: + +- You must have at least the [Developer role](../../permissions.md) for a group. + +For manually scheduled iteration cadences, you create and add iterations yourself. To create an iteration: -1. In a group, go to **{issues}** **Issues > Iterations**. -1. Click **New iteration**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. +1. Select **New iteration**. 1. Enter the title, a description (optional), a start date, and a due date. -1. Click **Create iteration**. The iteration details page opens. +1. Select **Create iteration**. The iteration details page opens. ## Edit an iteration > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218277) in GitLab 13.2. -NOTE: -You need Developer [permissions](../../permissions.md) or higher to edit an iteration. +Prerequisites: -To edit an iteration, click the three-dot menu (**{ellipsis_v}**) > **Edit iteration**. +- You must have at least the [Developer role](../../permissions.md) for a group. + +To edit an iteration, select the three-dot menu (**{ellipsis_v}**) > **Edit iteration**. ## Add an issue to an iteration @@ -76,7 +130,7 @@ The report also shows a breakdown of total issues in an iteration. Open iteration reports show a summary of completed, unstarted, and in-progress issues. Closed iteration reports show the total number of issues completed by the due date. -To view an iteration report, go to the iterations list page and click an iteration's title. +To view an iteration report, go to the iterations list page and select an iteration's title. ### Iteration burndown and burnup charts @@ -99,13 +153,15 @@ and get a more accurate understanding of scope attributable to each label. To group issues by label: +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. 1. In the **Group by** dropdown, select **Label**. 1. Select the **Filter by label** dropdown. 1. Select the labels you want to group by in the labels dropdown. You can also search for labels by typing in the search input. -1. Click or tap outside of the label dropdown. The page is now grouped by the selected labels. +1. Select or tap outside of the label dropdown. The page is now grouped by the selected labels. -## Disable iterations **(PREMIUM SELF)** +## Enable or disable iterations **(PREMIUM SELF)** GitLab Iterations feature is deployed with a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) @@ -129,6 +185,25 @@ Feature.disable(:group_iterations) Feature.disable(:group_iterations, Group.find(<group ID>)) ``` +### Enable or disable iteration cadences **(PREMIUM SELF)** + +Iteration Cadences feature is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:iteration_cadences) +``` + +To disable it: + +```ruby +Feature.disable(:iteration_cadences) +``` + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index b9f94d96b48..054c6ab7a6b 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -41,7 +41,7 @@ To see the latest code coverage for each project in your group: 1. In the **Latest test coverage results** section, use the **Select projects** dropdown to choose the projects you want to check. You can download code coverage data for specific projects using -[code coverage history](../../../ci/pipelines/settings.md#code-coverage-history). +[code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). ## Download historic test coverage data diff --git a/doc/user/group/saml_sso/img/saml_group_links_v13_9.png b/doc/user/group/saml_sso/img/saml_group_links_v13_9.png Binary files differindex 9bd2473f90c..1e0aa131aad 100644 --- a/doc/user/group/saml_sso/img/saml_group_links_v13_9.png +++ b/doc/user/group/saml_sso/img/saml_group_links_v13_9.png diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 8a5cdb79186..c3b57018154 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Introduced in GitLab 11.0. This page describes SAML for Groups. For instance-wide SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -[View the differences between SaaS and Self-Managed Authentication and Authorization Options](../../../administration/auth/README.md#saas-vs-self-managed-comparison). +[View the differences between SaaS and Self-Managed Authentication and Authorization Options](../../../administration/auth/index.md#saas-vs-self-managed-comparison). SAML on GitLab.com allows users to sign in through their SAML identity provider. If the user is not already a member, the sign-in process automatically adds the user to the appropriate group. @@ -330,11 +330,11 @@ Ensure your SAML identity provider sends an attribute statement named `Groups` o NOTE: To inspect the SAML response, you can use one of these [SAML debugging tools](#saml-debugging-tools). -Also note that the value for `Groups` or `groups` in the SAML reponse can be either the group name or +Also note that the value for `Groups` or `groups` in the SAML reponse can be either the group name or the group ID depending what the IdP sends to GitLab. When SAML SSO is enabled for the top-level group, `Maintainer` and `Owner` level users -see a new menu item in group **Settings > SAML Group Links**. You can configure one or more **SAML Group Links** to map +see a new menu item in group **Settings > SAML Group Links**. You can configure one or more **SAML Group Links** to map a SAML identity provider group name to a GitLab Access Level. This can be done for the parent group or the subgroups. To link the SAML groups from the `saml:AttributeStatement` example above: diff --git a/doc/user/group/settings/img/import_panel_v13_4.png b/doc/user/group/settings/img/import_panel_v13_4.png Binary files differdeleted file mode 100644 index e4e5b0e91a1..00000000000 --- a/doc/user/group/settings/img/import_panel_v13_4.png +++ /dev/null diff --git a/doc/user/group/settings/img/import_panel_v14_1.png b/doc/user/group/settings/img/import_panel_v14_1.png Binary files differnew file mode 100644 index 00000000000..28417383b6c --- /dev/null +++ b/doc/user/group/settings/img/import_panel_v14_1.png diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md index c097790ef16..5f732bee03f 100644 --- a/doc/user/group/settings/import_export.md +++ b/doc/user/group/settings/import_export.md @@ -70,7 +70,7 @@ For more details on the specific data persisted in a group export, see the  -1. After the export is generated, you should receive an e-mail with a link to the [exported contents](#exported-contents) +1. After the export is generated, you should receive an email with a link to the [exported contents](#exported-contents) in a compressed tar archive, with contents in NDJSON format. 1. Alternatively, you can come back to the project settings and download the @@ -84,7 +84,7 @@ As an administrator, you can modify the maximum import file size. To do so, use You can export groups from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. -The Enterprise Edition retains some group data that isn't part of the Community Edition. If you're exporting a group from the Enterprise Edition to the Community Edition, you may lose this data. For more information, see [downgrading from EE to CE](../../../README.md). +The Enterprise Edition retains some group data that isn't part of the Community Edition. If you're exporting a group from the Enterprise Edition to the Community Edition, you may lose this data. For more information, see [downgrading from EE to CE](../../../index.md). ## Importing the group @@ -93,9 +93,9 @@ on an existing group's page.  -1. On the New Group page, select the **Import group** tab. +1. On the New Group page, select the **Import group**. -  +  1. Enter your group name. diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 4532a391eef..7d674b5deac 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -125,7 +125,7 @@ When you add a member to a group, that member is also added to all subgroups. Permission level is inherited from the group's parent. This model allows access to subgroups if you have membership in one of its parents. -Jobs for pipelines in subgroups can use [runners](../../../ci/runners/README.md) registered to the parent group(s). +Jobs for pipelines in subgroups can use [runners](../../../ci/runners/index.md) registered to the parent group(s). This means secrets configured for the parent group are available to subgroup jobs. In addition, maintainers of projects that belong to subgroups can see the details of runners registered to parent group(s). diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index c1dd363c313..773d41947e2 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -65,7 +65,8 @@ To filter results: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 12.4. -GitLab provides the ability to filter analytics based on a date range. Data is shown for workflow items created during the selected date range. To filter results: +GitLab provides the ability to filter analytics based on a date range. +Data is shown for workflow items created during the selected date range. To filter results: 1. Select a group. 1. Optionally select a project. @@ -82,13 +83,16 @@ The "Time" metrics near the top of the page are measured as follows: The "Recent Activity" metrics near the top of the page are measured as follows: - **New Issues:** the number of issues created in the date range. -- **Deploys:** the number of deployments to production (1) in the date range. -- **Deployment Frequency:** the average number of deployments to production (1) per day in the date range. +- **Deploys:** the number of deployments <sup>1</sup> to production <sup>2</sup> in the date range. +- **Deployment Frequency:** the average number of deployments <sup>1</sup> to production <sup>2</sup> + per day in the date range. -(1) To give a more accurate representation of deployments that actually completed successfully, -the calculation for these two metrics changed in GitLab 13.9 from using the time a deployment was -created to the time a deployment finished. If you were referencing this metric prior to 13.9, please -keep this slight change in mind. +1. To give a more accurate representation of deployments that actually completed successfully, + the calculation for these two metrics changed in GitLab 13.9 from using the time a deployment was + created to the time a deployment finished. If you were referencing this metric prior to 13.9, please + keep this slight change in mind. +1. To see deployment metrics, you must have a + [production environment configured](../../../ci/environments/index.md#deployment-tier-of-environments). You can learn more about these metrics in our [analytics definitions](../../analytics/index.md). @@ -96,16 +100,16 @@ You can learn more about these metrics in our [analytics definitions](../../anal ## How the stages are measured -Value Stream Analytics measures each stage from its start event to its stop event. +Value Stream Analytics measures each stage from its start event to its end event. For example, a stage might start when one label is added to an issue, and end when another label is added. -Value Stream Analytics excludes work in progress, meaning it ignores any items that have not reached the stop event. +Value Stream Analytics excludes work in progress, meaning it ignores any items that have not reached the end event. Each stage of Value Stream Analytics is further described in the table below. | **Stage** | **Description** | | --------- | --------------- | | Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label is tracked only if it already has an [Issue Board list](../../project/issue_board.md) created for it. | -| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. | +| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (for example, `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. | | Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the closing pattern is not present, then the calculation takes the creation time of the first commit in the merge request as the start time. | | Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. | | Review | Measures the median time taken to review the merge request that has a closing issue pattern, between its creation and until it's merged. | @@ -121,7 +125,7 @@ How this works, behind the scenes: by the UI - default is 90 days). So it prohibits these pairs from being considered. 1. For the remaining `<issue, merge request>` pairs, we check the information that we need for the stages, like issue creation date, merge request merge time, - etc. + and so on. To sum up, anything that doesn't follow [GitLab flow](../../../topics/gitlab_flow.md) is not tracked and the Value Stream Analytics dashboard does not present any data for: @@ -133,7 +137,7 @@ Value Stream Analytics dashboard does not present any data for: ## How the production environment is identified Value Stream Analytics identifies production environments by looking for project -[environments](../../../ci/yaml/README.md#environment) with a name matching any of these patterns: +[environments](../../../ci/yaml/index.md#environment) with a name matching any of these patterns: - `prod` or `prod/*` - `production` or `production/*` @@ -144,7 +148,7 @@ You can change the name of a project environment in your GitLab CI/CD configurat ## Example workflow -Below is a simple fictional workflow of a single cycle that happens in a +Below is an example workflow of a single cycle that happens in a single day through all noted stages. Note that if a stage does not include a start and a stop time, its data is not included in the median time. It is assumed that milestones are created and a CI for testing and setting environments is configured. @@ -159,10 +163,11 @@ environments is configured. 12:00. 1. Make a second commit to the branch which mentions the issue number at 12.30 (stop of **Plan** stage / start of **Code** stage). -1. Push branch and create a merge request that contains the [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) +1. Push branch and create a merge request that contains the + [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) in its description at 14:00 (stop of **Code** stage / start of **Test** and **Review** stages). -1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../../ci/yaml/README.md) and +1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../../ci/yaml/index.md) and takes 5min (stop of **Test** stage). 1. Review merge request, ensure that everything is OK and merge the merge request at 19:00. (stop of **Review** stage / start of **Staging** stage). @@ -185,7 +190,7 @@ A few notes: commit doesn't mention the issue number, you can do this later in any commit of the branch you are working on. - You can see that the **Test** stage is not calculated to the overall time of - the cycle since it is included in the **Review** process (every MR should be + the cycle, because it is included in the **Review** process (every MR should be tested). - The example above was just **one cycle** of the seven stages. Add multiple cycles, calculate their median time and the result is what the dashboard of @@ -346,7 +351,7 @@ In this example, we'd like to measure times for deployment from a staging enviro After you create a value stream, you can customize it to suit your purposes. To edit a value stream: 1. Go to your group and select **Analytics > Value Stream**. -1. Find and select the relevant value stream from the value stream dropdown. +1. Find and select the relevant value stream from the value stream dropdown. 1. Next to the value stream dropdown, select **Edit**. The edit form is populated with the value stream details. 1. Optional: @@ -381,7 +386,7 @@ This chart visually depicts the average number of days it takes for cycles to be This chart uses the global page filters for displaying data based on the selected group, projects, and time frame. In addition, specific stages can be selected -from within the chart itself. +from the chart itself. The chart data is limited to the last 500 items. diff --git a/doc/user/img/todos_add_todo_sidebar.png b/doc/user/img/todos_add_todo_sidebar.png Binary files differdeleted file mode 100644 index aefec7a2d9c..00000000000 --- a/doc/user/img/todos_add_todo_sidebar.png +++ /dev/null diff --git a/doc/user/img/todos_add_todo_sidebar_v14_1.png b/doc/user/img/todos_add_todo_sidebar_v14_1.png Binary files differnew file mode 100644 index 00000000000..65120beca29 --- /dev/null +++ b/doc/user/img/todos_add_todo_sidebar_v14_1.png diff --git a/doc/user/img/todos_mark_done_sidebar.png b/doc/user/img/todos_mark_done_sidebar.png Binary files differdeleted file mode 100644 index 2badd880b40..00000000000 --- a/doc/user/img/todos_mark_done_sidebar.png +++ /dev/null diff --git a/doc/user/img/todos_mark_done_sidebar_v14_1.png b/doc/user/img/todos_mark_done_sidebar_v14_1.png Binary files differnew file mode 100644 index 00000000000..628fe65a7c1 --- /dev/null +++ b/doc/user/img/todos_mark_done_sidebar_v14_1.png diff --git a/doc/user/index.md b/doc/user/index.md index ab159cecdef..8fc91ec95ea 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -31,7 +31,7 @@ For more information, see [All GitLab Features](https://about.gitlab.com/feature To get familiar with the concepts needed to develop code on GitLab, read the following articles: - [Demo: Mastering Code Review With GitLab](https://about.gitlab.com/blog/2017/03/17/demo-mastering-code-review-with-gitlab/). -- [GitLab Workflow: An Overview](https://about.gitlab.com/topics/version-control/what-is-gitlab-workflow/#gitlab-workflow-a-use-case-scenario). +- [What is GitLab Flow?](https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/). - [Tutorial: It's all connected in GitLab](https://about.gitlab.com/blog/2016/03/08/gitlab-tutorial-its-all-connected/): an overview on code collaboration with GitLab. - [Trends in Version Control Land: Microservices](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). - [Trends in Version Control Land: Innersourcing](https://about.gitlab.com/topics/version-control/what-is-innersource/). @@ -46,7 +46,7 @@ GitLab is a Git-based platform that integrates a great number of essential tools - Organizing and prioritizing with [Issue Boards](project/issue_board.md). - Reviewing code in [Merge Requests](project/merge_requests/index.md) with live-preview changes per branch with [Review Apps](../ci/review_apps/index.md). -- Building, testing, and deploying with built-in [Continuous Integration](../ci/README.md). +- Building, testing, and deploying with built-in [Continuous Integration](../ci/index.md). - Deploying personal and professional static websites with [GitLab Pages](project/pages/index.md). - Integrating with Docker by using [GitLab Container Registry](packages/container_registry/index.md). - Tracking the development lifecycle by using [GitLab Value Stream Analytics](analytics/value_stream_analytics.md). @@ -64,7 +64,7 @@ With GitLab Enterprise Edition, you can also: - Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Search](search/advanced_search.md) for faster, more advanced code search across your entire GitLab instance. - [Authenticate users with Kerberos](../integration/kerberos.md). - [Mirror a repository](project/repository/repository_mirroring.md) from elsewhere on your local server. -- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipelines.md). +- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/pipelines/multi_project_pipelines.md). - [Lock files](project/file_lock.md) to prevent conflicts. - View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](project/deploy_boards.md). - Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). @@ -136,7 +136,7 @@ merge requests, code snippets, and commits. When performing inline reviews to implementations to your codebase through merge requests you can -gather feedback through [resolvable threads](discussions/index.md#resolvable-comments-and-threads). +gather feedback through [resolvable threads](discussions/index.md#resolve-a-thread). ### GitLab Flavored Markdown (GFM) @@ -164,7 +164,7 @@ you have quick access to. You can also gather feedback on them through ## GitLab CI/CD -Use built-in [GitLab CI/CD](../ci/README.md) to test, build, and deploy your applications +Use built-in [GitLab CI/CD](../ci/index.md) to test, build, and deploy your applications directly from GitLab. No third-party integrations needed. ## Features behind feature flags @@ -189,7 +189,7 @@ POST request with data to the webhook URL. ## API -Automate GitLab via [API](../api/README.md). +Automate GitLab via [API](../api/index.md). ## Git and GitLab diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md new file mode 100644 index 00000000000..35af316d7ac --- /dev/null +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -0,0 +1,94 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# New GKE cluster through IaC + +Learn how to create a new cluster on Google Kubernetes Engine (GKE) through +[Infrastructure as Code (IaC)](../../index.md). + +This process combines the GitLab Terraform and Google Terraform providers +with Kubernetes to help you create GKE clusters and deploy them through +GitLab. + +This document describes how to set up a [group-level cluster](../../../group/clusters/index.md) on GKE by importing an example project to get you started. +You can then modify the project files according to your needs. + +**Prerequisites:** + +- A GitLab group. +- A GitLab user with the Maintainer role in the group. +- A [GitLab personal access token](../../../profile/personal_access_tokens.md) with `api` access, created by a user with at least the Maintainer role in the group. +- A [Google Cloud Platform (GCP) service account](https://cloud.google.com/docs/authentication/getting-started). + +**Steps:** + +1. [Import the example project](#import-the-example-project). +1. [Add your GCP credentials to GitLab](#add-your-gcp-credentials-to-gitlab). +1. [Configure your project](#configure-your-project). +1. [Deploy your cluster](#deploy-your-cluster). + +## Import the example project + +Start by [importing the example project by URL](../../../project/import/repo_by_url.md). Use `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke.git` as URL. + +## Add your GCP credentials to GitLab + +After importing the project, you need to set up [CI environment variables](../../../../ci/variables/index.md) to associate your cluster on GCP to your group in GitLab. + +We advise that you [set environment variables through the GitLab UI](../../../../ci/variables/index.md#add-a-cicd-variable-to-a-project) +so that your credentials are not exposed through the code. To do so, follow the steps below. + +### Prepare your credentials on GCP + +1. Create a [GCP service account](https://cloud.google.com/docs/authentication/getting-started) to authenticate GCP with GitLab. It needs the following roles: `Computer Network Viewer`, `Kubernetes Engine Admin`, and `Service Account User`. +1. Download the JSON file with the service account key. +1. On your computer, encode the JSON file to `base64` (replace `/path/to/sa-key.json` to the path to your key): + + ```shell + base64 /path/to/sa-key.json | tr -d \\n` + ``` + +1. Use the output of this command as the `BASE64_GOOGLE_CREDENTIALS` environment variable in the next step. + +### Add your credentials to GitLab as environment variables + +1. On GitLab, from your project's sidebar, go to **Settings > CI/CD** and expand **Variables**. +1. Add your `GITLAB_TOKEN` ([personal access token](../../../profile/personal_access_tokens.md)). +1. Add the variable `BASE64_GOOGLE_CREDENTIALS` from the previous step. + +## Configure your project + +After authenticating with GCP, replace the project's defaults from the example +project with your own. To do so, edit the files as described below. + +Edit `gke.tf`: + +1. **(Required)** Replace the GCP `project` with a unique project name. +1. **(Optional)** Choose the `name` of your cluster. +1. **(Optional)** Choose the `region` and `zone` that you would like to deploy your cluster to. +1. Push the changes to your project's default branch. + +Edit `group_cluster.tf`: + +1. **(Required)**: Replace the `full_path` with the path to your group. +1. **(Optional)**: Choose your cluster base domain through `domain`. +1. **(Optional)**: Choose your environment through `environment_scope`. +1. Push the changes to your project's default branch. + +Refer to the [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) and the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) documentation for further resource options. + +## Deploy your cluster + +After adjusting the files in the previous step, manually trigger the deployment of your cluster. In GitLab: + +1. From your project's sidebar, go to **CI/CD > Pipelines**. +1. Select the dropdown icon (**{angle-down}**) next to the play icon (**{play}**). +1. Select **deploy** to manually trigger the deployment job. + +When the pipeline finishes successfully, you can see your new cluster: + +- In GCP: on your [GCP console's Kubernetes list](https://console.cloud.google.com/kubernetes/list). +- In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md new file mode 100644 index 00000000000..d5840641aab --- /dev/null +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -0,0 +1,69 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Inventory object **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0. + +An inventory object is a `ConfigMap` object for keeping track of the set of objects applied to a cluster. +When you remove objects from a manifest repository, GitLab Kubernetes Agent uses a corresponding inventory object to +prune (delete) objects from the cluster. + +The GitLab Kubernetes Agent creates an inventory object for each manifest project specified in the +`gitops.manifest_projects` configuration section. The inventory object has to be stored somewhere in the cluster. +The default behavior is: + +- The `namespace` used comes from `gitops.manifest_projects[].default_namespace`. If you don't specify this parameter + explicitly, the inventory object is stored in the `default` namespace. +- The `name` is generated from the numeric project ID of the manifest project and the numeric agent ID. + + This way the GitLab Kubernetes Agent constructs the name and local where the inventory object is + stored in the cluster. + +The GitLab Kubernetes Agent cannot locate the existing inventory object if you: + +- Change `gitops.manifest_projects[].default_namespace` parameter. +- Move manifests into another project. + +## Inventory object template + +The inventory object template is a `ConfigMap` object that allows you to configure the namespace and the name of the inventory +object. Store this template with manifest files in a single group. + +Example inventory object template: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: unique-name-for-the-inventory + namespace: my-project-namespace + labels: + cli-utils.sigs.k8s.io/inventory-id: unique-name-for-the-inventory +``` + +- The `namespace` and `name` fields configure where the real inventory object is created. +- The `cli-utils.sigs.k8s.io/inventory-id` label with its corresponding value is set on the inventory object, created + from this template. Make sure that the value is unique (for example, a string of random characters) and doesn't clash + with any existing or future inventory object templates. +- Objects tracked by this inventory object have the `config.k8s.io/owning-inventory` annotation set to the value of + the `cli-utils.sigs.k8s.io/inventory-id` label. +- The label's value doesn't have to match the `name` but it's convenient to have them set to the same value. +- Make sure that the `name` is unique so that it doesn't conflict with another inventory object in the same + namespace in the future. + +## Using GitOps with pre-existing Kubernetes objects + +The GitLab Kubernetes Agent treats manifest files in the manifest repository as the source of truth. When it applies +objects from the files to the cluster, it tracks them in an inventory object. If an object already exists, +GitLab Kubernetes Agent behaves differently based on the `gitops.manifest_projects[].inventory_policy` configuration. +Check the table below with the available options and when to use them. + +`inventory_policy` value | Description | +------------------------ | ------------------------------------------------------------------------------------------- | +`must_match` | This is the default policy. A live object must have the `config.k8s.io/owning-inventory` annotation set to the same value as the `cli-utils.sigs.k8s.io/inventory-id` label on the corresponding inventory object to be updated. Object is not updated and an error is reported if the values don't match or the object doesn't have the annotation. | +`adopt_if_no_inventory` | This mode allows to "adopt" an object if it doesn't have the `config.k8s.io/owning-inventory` annotation. Use this mode if you want to start managing existing objects using the GitOps feature. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | +`adopt_all` | This mode allows to "adopt" an object even if it has the `config.k8s.io/owning-inventory` annotation set to a different value. This mode can be useful if you want to migrate a set of objects from one agent to another one or from some other tool to the GitLab Kubernetes Agent. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 05ffab93f85..bdaae4b8225 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -16,7 +16,7 @@ GitLab, and support Terraform best practices. ## Quick Start Use the following `.gitlab-ci.yml` to set up a basic Terraform project integration -for GitLab versions 13.5 and later: +for GitLab versions 14.0 and later: ```yaml include: @@ -38,7 +38,7 @@ This template includes some opinionated decisions, which you can override: - Creating [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml): `init`, `validate`, `build`, and `deploy`. These stages [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) - `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on `master`. + `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on the default branch. This video from January 2021 walks you through all the GitLab Terraform integration features: @@ -89,7 +89,7 @@ tools or rely on 3rd party solutions to streamline their IaC workflows. Read more on setting up and [using the merge request integrations](mr_integration.md). -## The GitLab terraform provider +## The GitLab Terraform provider WARNING: The GitLab Terraform provider is released separately from GitLab. @@ -101,3 +101,39 @@ owned by GitLab, where everyone can contribute. The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) is available as part of the official Terraform provider documentations. + +## Create a new cluster through IaC + +Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](clusters/connect/new_gke_cluster.md). + +## Troubleshooting + +### `gitlab_group_share_group` resources not detected when subgroup state is refreshed + +The GitLab Terraform provider can fail to detect existing `gitlab_group_share_group` resources +due to the issue ["User with permissions cannot retrieve `share_with_groups` from the API"](https://gitlab.com/gitlab-org/gitlab/-/issues/328428). +This results in an error when running `terraform apply` because Terraform attempts to recreate an +existing resource. + +For example, consider the following group/subgroup configuration: + +```plaintext +parent-group +├── subgroup-A +└── subgroup-B +``` + +Where: + +- User `user-1` creates `parent-group`, `subgroup-A`, and `subgroup-B`. +- `subgroup-A` is shared with `subgroup-B`. +- User `terraform-user` is member of `parent-group` with inherited `owner` access to both subgroups. + +When the Terraform state is refreshed, the API query `GET /groups/:subgroup-A_id` issued by the provider does not return the +details of `subgroup-B` in the `shared_with_groups` array. This leads to the error. + +To workaround this issue, make sure to apply one of the following conditions: + +1. The `terraform-user` creates all subgroup resources. +1. Grant Maintainer or Owner role to the `terraform-user` user on `subgroup-B`. +1. The `terraform-user` inherited access to `subgroup-B` and `subgroup-B` contains at least one project. diff --git a/doc/user/infrastructure/mr_integration.md b/doc/user/infrastructure/mr_integration.md index 6f8b1d8d569..66e00bab6ce 100644 --- a/doc/user/infrastructure/mr_integration.md +++ b/doc/user/infrastructure/mr_integration.md @@ -10,7 +10,7 @@ Collaborating around Infrastructure as Code (IaC) changes requires both code cha ## Output Terraform Plan information into a merge request -Using the [GitLab Terraform Report artifact](../../ci/yaml/README.md#artifactsreportsterraform), +Using the [GitLab Terraform Report artifact](../../ci/yaml/index.md#artifactsreportsterraform), you can expose details from `terraform plan` runs directly into a merge request widget, enabling you to see statistics about the resources that Terraform creates, modifies, or destroys. @@ -57,7 +57,7 @@ To manually configure a GitLab Terraform Report artifact requires the following 1. Define a `script` that runs `terraform plan` and `terraform show`. These commands pipe the output and convert the relevant bits into a store variable `PLAN_JSON`. This JSON is used to create a - [GitLab Terraform Report artifact](../../ci/yaml/README.md#artifactsreportsterraform). + [GitLab Terraform Report artifact](../../ci/yaml/index.md#artifactsreportsterraform). The Terraform report obtains a Terraform `tfplan.json` file. The collected Terraform plan report is uploaded to GitLab as an artifact, and is shown in merge requests. diff --git a/doc/user/infrastructure/terraform_state.md b/doc/user/infrastructure/terraform_state.md index 0b92ea46338..57db2b47de4 100644 --- a/doc/user/infrastructure/terraform_state.md +++ b/doc/user/infrastructure/terraform_state.md @@ -242,7 +242,7 @@ An example setup is shown below: Outputs from the data source can now be referenced in your Terraform resources using `data.terraform_remote_state.example.outputs.<OUTPUT-NAME>`. -You need at least [developer access](../permissions.md) to the target project +You need at least the [Developer role](../permissions.md) in the target project to read the Terraform state. ## Migrating to GitLab Managed Terraform state diff --git a/doc/user/markdown.md b/doc/user/markdown.md index fdfd953e52a..fc278007463 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -31,7 +31,7 @@ and the [main GitLab website](https://about.gitlab.com) use [Kramdown](https://k You should not view this page in the documentation, but instead [view these styles as they appear on GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md). GitLab Flavored Markdown extends the [CommonMark specification](https://spec.commonmark.org/current/). -It was inspired by [GitHub Flavored Markdown](https://docs.github.com/en/github/writing-on-github/basic-writing-and-formatting-syntax). +It was inspired by [GitHub Flavored Markdown](https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax). ## Where you can use GitLab Flavored Markdown @@ -412,6 +412,10 @@ A table of contents is an unordered list that links to subheadings in the docume To add a table of contents to a Markdown file, wiki page, issue request, or merge request description, add the `[[_TOC_]]` tag on its own line. +NOTE: +You can add a table of contents to issues and merge requests, but you can't add one +to notes or comments. + ```markdown This is an intro sentence to my Wiki page. @@ -507,7 +511,8 @@ This example links to `<wiki_root>/miscellaneous.md`: GitLab Flavored Markdown renders GitLab-specific references. For example, you can reference an issue, a commit, a team member, or even an entire project team. GitLab Flavored Markdown turns -that reference into a link so you can navigate between them. +that reference into a link so you can navigate between them. All references to projects should use the +**project slug** rather than the project name. Additionally, GitLab Flavored Markdown recognizes certain cross-project references and also has a shorthand version to reference other projects from the same namespace. @@ -519,7 +524,7 @@ GitLab Flavored Markdown recognizes the following: | specific user | `@user_name` | | | | specific group | `@group_name` | | | | entire team | `@all` | | | -| project | `namespace/project>` | | | +| project | `namespace/project` | | | | issue | ``#123`` | `namespace/project#123` | `project#123` | | merge request | `!123` | `namespace/project!123` | `project!123` | | snippet | `$123` | `namespace/project$123` | `project$123` | @@ -1181,7 +1186,7 @@ Do not edit the following codeblock. It uses HTML to skip the Vale ReferenceLink <pre class="highlight"><code>- This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) -- This is a [relative link to a readme one directory higher](../README.md) +- This is a [relative link to a readme one directory higher](../index.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") Using header ID anchors: @@ -1204,7 +1209,7 @@ Some text to show that the reference links can follow later. - This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) -- This is a [relative link to a README one directory higher](../README.md) +- This is a [relative link to a README one directory higher](../index.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") Using header ID anchors: diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index b5170dfa55b..b6eb2975374 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -293,7 +293,7 @@ To install a package: WARNING: Never commit the `auth.json` file to your repository. To install packages from a CI/CD job, consider using the [`composer config`](https://getcomposer.org/doc/articles/handling-private-packages.md#satis) tool with your personal access token -stored in a [GitLab CI/CD variable](../../../ci/variables/README.md) or in +stored in a [GitLab CI/CD variable](../../../ci/variables/index.md) or in [HashiCorp Vault](../../../ci/secrets/index.md). ## Supported CLI commands diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index a429e746cf2..c6cc7e7905a 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -269,7 +269,7 @@ conan upload Hello/0.1@mycompany/beta --all > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11678) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.7. -To work with Conan commands in [GitLab CI/CD](../../../ci/README.md), you can +To work with Conan commands in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. You can provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each Conan diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index d6e86e64e78..eecc17fd60d 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -135,7 +135,7 @@ To view these commands, go to your project's **Packages & Registries > Container ## Build and push by using GitLab CI/CD -Use [GitLab CI/CD](../../../ci/yaml/README.md) to build and push images to the +Use [GitLab CI/CD](../../../ci/yaml/index.md) to build and push images to the Container Registry. Use it to test, build, and deploy your project from the Docker image you created. @@ -154,7 +154,7 @@ To use CI/CD to authenticate, you can use: docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY ``` -- A [CI job token](../../../ci/triggers/README.md#ci-job-token). +- A [CI job token](../../../ci/triggers/index.md#ci-job-token). ```shell docker login -u $CI_JOB_USER -p $CI_JOB_TOKEN $CI_REGISTRY @@ -210,7 +210,7 @@ build: - docker push $CI_REGISTRY/group/project/image:latest ``` -You can also make use of [other CI/CD variables](../../../ci/variables/README.md) to avoid hard-coding: +You can also make use of [other CI/CD variables](../../../ci/variables/index.md) to avoid hard-coding: ```yaml build: @@ -309,7 +309,7 @@ in addition to the steps in the [Docker-in-Docker](../../../ci/docker/using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker) section: 1. Update the `image` and `service` to point to your registry. -1. Add a service [alias](../../../ci/yaml/README.md#servicesalias). +1. Add a service [alias](../../../ci/yaml/index.md#servicesalias). Below is an example of what your `.gitlab-ci.yml` should look like: @@ -332,6 +332,36 @@ If you forget to set the service alias, the `docker:19.03.12` image is unable to error during connect: Get http://docker:2376/v1.39/info: dial tcp: lookup docker on 192.168.0.1:53: no such host ``` +### Using a Docker-in-Docker image with Dependency Proxy + +To use your own Docker images with Dependency Proxy, follow these steps +in addition to the steps in the +[Docker-in-Docker](../../../ci/docker/using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker) section: + +1. Update the `image` and `service` to point to your registry. +1. Add a service [alias](../../../ci/yaml/index.md#servicesalias). + +Below is an example of what your `.gitlab-ci.yml` should look like: + +```yaml +build: + image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:19.03.12 + services: + - name: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:18.09.7-dind + alias: docker + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` + +If you forget to set the service alias, the `docker:19.03.12` image is unable to find the +`dind` service, and an error like the following is thrown: + +```plaintext +error during connect: Get http://docker:2376/v1.39/info: dial tcp: lookup docker on 192.168.0.1:53: no such host +``` + ## Delete images You can delete images from your Container Registry in multiple ways. @@ -533,7 +563,7 @@ You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the To create a cleanup policy in the UI: -1. For your project, go to **Settings > CI/CD**. +1. For your project, go to **Settings > Packages & Registries**. 1. Expand the **Clean up image tags** section. 1. Complete the fields. @@ -788,7 +818,7 @@ these steps: the tags' names will be in the `list_o_tags.out` file: ```shell - # Get a list of all tags in a certain container repository while considering [pagination](../../../api/README.md#pagination) + # Get a list of all tags in a certain container repository while considering [pagination](../../../api/index.md#pagination) echo -n "" > list_o_tags.out; for i in {1..N}; do curl --header 'PRIVATE-TOKEN: <PAT>' "https://gitlab.example.com/api/v4/projects/<Project_id>/registry/repositories/<container_repo_id>/tags?per_page=100&page=${i}" | jq '.[].name' | sed 's:^.\(.*\).$:\1:' >> list_o_tags.out; done ``` diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md new file mode 100644 index 00000000000..59213ccb1a0 --- /dev/null +++ b/doc/user/packages/debian_repository/index.md @@ -0,0 +1,111 @@ +--- +stage: Package +group: Package +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Debian packages in the Package Registry **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5835) in GitLab 14.1. + +WARNING: +The Debian package registry for GitLab is under development and isn't ready for production use due to +limited functionality. + +Publish Debian packages in your project's Package Registry. Then install the +packages whenever you need to use them as a dependency. + +Project and Group packages are supported. + +For documentation of the specific API endpoints that Debian package manager +clients use, see the [Debian API documentation](../../../api/packages/debian.md). + +## Enable Debian repository feature + +Debian repository support is still a work in progress. It's gated behind a feature flag that's +**disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to enable it. + +To enable it: + +```ruby +Feature.enable(:debian_packages) +``` + +To disable it: + +```ruby +Feature.disable(:debian_packages) +``` + +## Build a Debian package + +Creating a Debian package is documented [on the Debian Wiki](https://wiki.debian.org/Packaging). + +## Create a Distribution + +On the project-level, Debian packages are published using *Debian Distributions*. To publish +packages on the group level, create a distribution with the same `codename`. + +To create a project-level distribution: + +```shell +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/<project_id>/debian_distributions?codename=unstable +``` + +Example response: + +```json +{ + "id": 1, + "codename": "unstable", + "suite": null, + "origin": null, + "label": null, + "version": null, + "description": null, + "valid_time_duration_seconds": null, + "components": [ + "main" + ], + "architectures": [ + "all", + "amd64" + ] +} +``` + +More information on Debian distribution APIs: + +- [Debian project distributions API](../../../api/packages/debian_project_distributions.md) +- [Debian group distributions API](../../../api/packages/debian_group_distributions.md) + +## Publish a package + +Once built, several files are created: + +- `.deb` files: the binary packages +- `.udeb` files: lightened .deb files, used for Debian-Installer (if needed) +- `.tar.{gz,bz2,xz,...}` files: Source files +- `.dsc` file: Source metadata, and list of source files (with hashes) +- `.buildinfo` file: Used for Reproducible builds (optional) +- `.changes` file: Upload metadata, and list of uploaded files (all the above) + +To upload these files, you can use `dput-ng >= 1.32` (Debian bullseye): + +```shell +cat <<EOF > dput.cf +[gitlab] +method = https +fqdn = <login>:<your_access_token>@gitlab.example.com +incoming = /api/v4/projects/<project_id>/packages/debian +EOF + +dput --config=dput.cf --unchecked --no-upload-log gitlab <your_package>.changes +``` + +## Install a package + +The Debian package registry for GitLab is under development, and isn't ready for production use. You +cannot install packages from the registry. However, you can download files directly from the UI. diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 3dd900d2cbe..e2957aff756 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -123,7 +123,7 @@ Proxy manually without including the port: docker pull gitlab.example.com:443/my-group/dependency_proxy/containers/alpine:latest ``` -You can also use [custom CI/CD variables](../../../ci/variables/README.md#custom-cicd-variables) to store and access your personal access token or other valid credentials. +You can also use [custom CI/CD variables](../../../ci/variables/index.md#custom-cicd-variables) to store and access your personal access token or other valid credentials. ### Store a Docker image in Dependency Proxy cache @@ -134,13 +134,13 @@ To store a Docker image in Dependency Proxy storage: 1. Use one of these commands. In these examples, the image is `alpine:latest`. 1. You can also pull images by digest to specify exactly which version of an image to pull. - - Pull an image by tag by adding the image to your [`.gitlab-ci.yml`](../../../ci/yaml/README.md#image) file: + - Pull an image by tag by adding the image to your [`.gitlab-ci.yml`](../../../ci/yaml/index.md#image) file: ```shell image: gitlab.example.com/groupname/dependency_proxy/containers/alpine:latest ``` - - Pull an image by digest by adding the image to your [`.gitlab-ci.yml`](../../../ci/yaml/README.md#image) file: + - Pull an image by digest by adding the image to your [`.gitlab-ci.yml`](../../../ci/yaml/index.md#image) file: ```shell image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/alpine@sha256:c9375e662992791e3f39e919b26f510e5254b42792519c180aad254e6b38f4dc @@ -180,7 +180,7 @@ Watch how to [use the Dependency Proxy to help avoid Docker Hub rate limits](htt In November 2020, Docker introduced [rate limits on pull requests from Docker Hub](https://docs.docker.com/docker-hub/download-rate-limit/). -If your GitLab [CI/CD configuration](../../../ci/README.md) uses +If your GitLab [CI/CD configuration](../../../ci/index.md) uses an image from Docker Hub, each time a job runs, it may count as a pull request. To help get around this limit, you can pull your image from the Dependency Proxy cache instead. @@ -252,3 +252,22 @@ hub_docker_quota_check: - | TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq --raw-output .token) && curl --head --header "Authorization: Bearer $TOKEN" "https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest" 2>&1 ``` + +## Troubleshooting + +### Dependency Proxy Connection Failure + +If a service alias is not set the `docker:19.03.12` image is unable to find the +`dind` service, and an error like the following is thrown: + +```plaintext +error during connect: Get http://docker:2376/v1.39/info: dial tcp: lookup docker on 192.168.0.1:53: no such host +``` + +This can be resolved by setting a service alias for the Docker service: + +```yaml +services: + - name: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:18.09.7-dind + alias: docker +``` diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index e20ac57e64f..cb5258981be 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -20,14 +20,14 @@ Publish generic files, like release binaries, in your project's Package Registry ## Authenticate to the Package Registry -To authenticate to the Package Registry, you need either a [personal access token](../../../api/README.md#personalproject-access-tokens), -[CI/CD job token](../../../api/README.md#gitlab-cicd-job-token), or [deploy token](../../project/deploy_tokens/index.md). +To authenticate to the Package Registry, you need either a [personal access token](../../../api/index.md#personalproject-access-tokens), +[CI/CD job token](../../../api/index.md#gitlab-cicd-job-token), or [deploy token](../../project/deploy_tokens/index.md). In addition to the standard API authentication mechanisms, the generic package API allows authentication with HTTP Basic authentication for use with tools that do not support the other available mechanisms. The `user-id` is not checked and -may be any value, and the `password` must be either a [personal access token](../../../api/README.md#personalproject-access-tokens), -a [CI/CD job token](../../../api/README.md#gitlab-cicd-job-token), or a [deploy token](../../project/deploy_tokens/index.md). +may be any value, and the `password` must be either a [personal access token](../../../api/index.md#personalproject-access-tokens), +a [CI/CD job token](../../../api/index.md#gitlab-cicd-job-token), or a [deploy token](../../project/deploy_tokens/index.md). ## Publish a package file @@ -37,7 +37,7 @@ If a package with the same name, version, and filename already exists, it is als Prerequisites: -- You need to [authenticate with the API](../../../api/README.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. +- You need to [authenticate with the API](../../../api/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. ```plaintext PUT /projects/:id/packages/generic/:package_name/:package_version/:file_name?status=:status @@ -45,7 +45,7 @@ PUT /projects/:id/packages/generic/:package_name/:package_version/:file_name?sta | Attribute | Type | Required | Description | | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/index.md#namespaced-path-encoding). | | `package_name` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). | `package_version` | string | yes | The package version. The following regex validates this: `\A(\.?[\w\+-]+\.?)+\z`. You can test your version strings on [Rubular](https://rubular.com/r/aNCV0wG5K14uq8). | `file_name` | string | yes | The filename. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). @@ -77,7 +77,7 @@ If multiple packages have the same name, version, and filename, then the most re Prerequisites: -- You need to [authenticate with the API](../../../api/README.md#authentication). If authenticating with a deploy token, it must be configured with the `read_package_registry` and/or `write_package_registry` scope. +- You need to [authenticate with the API](../../../api/index.md#authentication). If authenticating with a deploy token, it must be configured with the `read_package_registry` and/or `write_package_registry` scope. ```plaintext GET /projects/:id/packages/generic/:package_name/:package_version/:file_name @@ -85,7 +85,7 @@ GET /projects/:id/packages/generic/:package_name/:package_version/:file_name | Attribute | Type | Required | Description | | -------------------| --------------- | ---------| ------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/index.md#namespaced-path-encoding). | | `package_name` | string | yes | The package name. | | `package_version` | string | yes | The package version. | | `file_name` | string | yes | The filename. | @@ -108,7 +108,7 @@ curl --user "user:<your_access_token>" \ ## Publish a generic package by using CI/CD -To work with generic packages in [GitLab CI/CD](../../../ci/README.md), you can use +To work with generic packages in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. For example: diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index 36348fcde18..0c04c4a23d0 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -142,7 +142,7 @@ the following documentation: - [Dependency Management in Go](../../../development/go_guide/dependencies.md) - [Go Modules Reference](https://golang.org/ref/mod) - [Documentation (`golang.org`)](https://golang.org/doc/) -- [Learn (`learn.go.dev`)](https://learn.go.dev/) +- [Learn (`go.dev/learn`)](https://go.dev/learn/) ### Set environment variables diff --git a/doc/user/packages/helm_repository/index.md b/doc/user/packages/helm_repository/index.md new file mode 100644 index 00000000000..26d8bf76cd6 --- /dev/null +++ b/doc/user/packages/helm_repository/index.md @@ -0,0 +1,89 @@ +--- +stage: Package +group: Package +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Helm charts in the Package Registry **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18997) in GitLab 14.1. + +WARNING: +The Helm package registry for GitLab is under development and isn't ready for production use due to +limited functionality. + +Publish Helm packages in your project's Package Registry. Then install the +packages whenever you need to use them as a dependency. + +For documentation of the specific API endpoints that Helm package manager +clients use, see the [Helm API documentation](../../../api/packages/helm.md). + +## Build a Helm package + +Creating a Helm package is documented [in the Helm documentation](https://helm.sh/docs/intro/using_helm/#creating-your-own-charts). + +## Authenticate to the Helm repository + +To authenticate to the Helm repository, you need either: + +- A [personal access token](../../../api/index.md#personalproject-access-tokens). +- A [deploy token](../../project/deploy_tokens/index.md). +- A [CI/CD job token](../../../api/index.md#gitlab-cicd-job-token). + +## Publish a package + +Once built, a chart can be uploaded to the `stable` channel with `curl` or `helm-push`: + +- With `curl`: + + ```shell + curl --request POST \ + --form 'chart=@mychart-0.1.0.tgz' \ + --user <username>:<personal_access_token> \ + https://gitlab.example.com/api/v4/projects/1/packages/helm/api/stable/charts + ``` + +- With the [`helm-push`](https://github.com/chartmuseum/helm-push/#readme) plugin: + + ```shell + helm repo add --username <username> --password <personal_access_token> project-1 https://gitlab.example.com/api/v4/projects/1/packages/helm/stable + helm push mychart-0.1.0.tgz project-1 + ``` + +## Use CI/CD to publish a Helm package + +To publish a Helm package automated through [GitLab CI/CD](../../../ci/index.md), you can use +`CI_JOB_TOKEN` in place of the personal access token in your commands. + +For example: + +```yaml +image: curlimages/curl:latest + +stages: + - upload + +upload: + stage: upload + script: + - 'curl --request POST --user gitlab-ci-token:$CI_JOB_TOKEN --form "chart=@mychart-0.1.0.tgz" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/stable/charts"' +``` + +## Install a package + +To install the latest version of a chart, use the following command: + +```shell +helm repo add --username <username> --password <personal_access_token> project-1 https://gitlab.example.com/api/v4/projects/1/packages/helm/stable +helm install my-release project-1/mychart +``` + +If the repo has previously been added, you may need to run: + +```shell +helm repo update +``` + +To update the Helm client with the most currently available charts. + +See [Using Helm](https://helm.sh/docs/intro/using_helm/) for more information. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index f0bf2fc3363..9158b5cc674 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -17,6 +17,7 @@ The Package Registry supports the following formats: | [Composer](composer_repository/index.md) | 13.2+ | | [Conan](conan_repository/index.md) | 12.6+ | | [Go](go_proxy/index.md) | 13.1+ | +| [Helm](helm_repository/index.md) | 14.1+ | | [Maven](maven_repository/index.md) | 11.3+ | | [npm](npm_registry/index.md) | 11.7+ | | [NuGet](nuget_repository/index.md) | 12.8+ | @@ -40,7 +41,6 @@ guides you through the process. | Conda | [#36891](https://gitlab.com/gitlab-org/gitlab/-/issues/36891) | | CRAN | [#36892](https://gitlab.com/gitlab-org/gitlab/-/issues/36892) | | Debian | [Draft: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50438) | -| Helm | [#18997](https://gitlab.com/gitlab-org/gitlab/-/issues/18997) | | Opkg | [#36894](https://gitlab.com/gitlab-org/gitlab/-/issues/36894) | | P2 | [#36895](https://gitlab.com/gitlab-org/gitlab/-/issues/36895) | | Puppet | [#36897](https://gitlab.com/gitlab-org/gitlab/-/issues/36897) | diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md index 00370bd2f48..86b85d0c9f5 100644 --- a/doc/user/packages/infrastructure_registry/index.md +++ b/doc/user/packages/infrastructure_registry/index.md @@ -35,7 +35,7 @@ documentation for your package type: ## Use GitLab CI/CD to build packages -To use [GitLab CI/CD](../../../ci/README.md) to build packages, you can +To use [GitLab CI/CD](../../../ci/index.md) to build packages, you can authenticate with the [`CI_JOB_TOKEN` predefined variable](../../../ci/variables/predefined_variables.md). CI/CD templates, which you can use to get started, are in [this repository](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 2567cc3b828..70b9c28da76 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -412,7 +412,7 @@ repositories { - The `PROJECT_ID` is your project ID, which you can view on your project's home page. - Replace `gitlab.example.com` with your domain name. - For retrieving artifacts, use either the - [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the project + [URL-encoded](../../../api/index.md#namespaced-path-encoding) path of the project (like `group%2Fproject`) or the project's ID (like `42`). However, only the project's ID can be used for publishing. @@ -471,7 +471,7 @@ repositories { - For `PROJECT_ID`, use your project ID, which you can view on your project's home page. - Replace `gitlab.example.com` with your domain name. - For retrieving artifacts, use either the - [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the group + [URL-encoded](../../../api/index.md#namespaced-path-encoding) path of the group (like `group%2Fsubgroup`) or the group's ID (like `12`). ### Instance-level Maven endpoint @@ -530,7 +530,7 @@ repositories { - The `PROJECT_ID` is your project ID, which you can view on your project's home page. - Replace `gitlab.example.com` with your domain name. - For retrieving artifacts, use either the - [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the project + [URL-encoded](../../../api/index.md#namespaced-path-encoding) path of the project (like `group%2Fproject`) or the project's ID (like `42`). However, only the project's ID can be used for publishing. @@ -745,7 +745,7 @@ You can create a new package each time the `master` branch is updated. <repositories> <repository> <id>gitlab-maven</id> - <url>$env{CI_API_V4_URL}/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>${env.CI_API_V4_URL}/projects/${env.CI_PROJECT_ID}/packages/maven</url> </repository> </repositories> <distributionManagement> @@ -813,6 +813,19 @@ You can play around with the regex and try your version strings on [this regular ## Troubleshooting +To improve performance, Maven caches files related to a package. If you encounter issues, clear +the cache with these commands: + +```shell +rm -rf ~/.m2/repository +``` + +If you're using Gradle, run this command to clear the cache: + +```shell +rm -rf ~/.gradle/caches +``` + ### Review network trace logs If you are having issues with the Maven Repository, you may want to review network trace logs. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 735873be237..1e5c294689b 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -90,7 +90,7 @@ To use the GitLab endpoint for npm packages, choose an option: - **Project-level**: Use when you have few npm packages and they are not in the same GitLab group. The [package naming convention](#package-naming-convention) is not enforced at this level. - Instead, you should use a [scope](https://docs.npmjs.com/cli/v6/using-npm/scope) for your package. + Instead, you should use a [scope](https://docs.npmjs.com/cli/v6/using-npm/scope/) for your package. When you use a scope, the registry URL is [updated](#authenticate-to-the-package-registry) only for that scope. - **Instance-level**: Use when you have many npm packages in different GitLab groups or in their own namespace. Be sure to comply with the [package naming convention](#package-naming-convention). @@ -205,7 +205,7 @@ Then, you can run `npm publish` either locally or by using GitLab CI/CD. NPM_TOKEN=<your_token> npm publish ``` -- **GitLab CI/CD:** Set an `NPM_TOKEN` [CI/CD variable](../../../ci/variables/README.md) +- **GitLab CI/CD:** Set an `NPM_TOKEN` [CI/CD variable](../../../ci/variables/index.md) under your project's **Settings > CI/CD > Variables**. ## Package naming convention @@ -287,7 +287,7 @@ Prerequisites: It must match exactly, including the case. This is different than the npm naming convention, but it is required to work with the GitLab Package Registry. -To work with npm commands within [GitLab CI/CD](../../../ci/README.md), you can use +To work with npm commands within [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. An example `.gitlab-ci.yml` file for publishing npm packages: @@ -311,8 +311,7 @@ step-by-step guide and demo project for a complete example. ## Configure the GitLab npm registry with Yarn 2 -You can get started with Yarn 2 by following the documentation at -[https://yarnpkg.com/getting-started/install](https://yarnpkg.com/getting-started/install). +You can get started with Yarn 2 by following the [Yarn documentation](https://yarnpkg.com/getting-started/install/). To publish and install with the project-level npm endpoint, set the following configuration in `.yarnrc.yml`: @@ -456,6 +455,14 @@ Due to a bug in npm 6.9.0, deleting distribution tags fails. Make sure your npm When troubleshooting npm issues, first run the same command with the `--verbose` flag to confirm what registry you are hitting. +To improve performance, npm caches files related to a package. Note that npm doesn't remove data by +itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with +this command: + +```shell +npm cache clean --force +``` + ### Error running Yarn with the Package Registry for npm registry If you are using [Yarn](https://classic.yarnpkg.com/en/) with the npm registry, you may get @@ -512,7 +519,7 @@ And the `.npmrc` file should look like: ### `npm install` returns `Error: Failed to replace env in config: ${npm_TOKEN}` -You do not need a token to run `npm install` unless your project is private. The token is only required to publish. If the `.npmrc` file was checked in with a reference to `$npm_TOKEN`, you can remove it. If you prefer to leave the reference in, you must set a value prior to running `npm install` or set the value by using [GitLab CI/CD variables](../../../ci/variables/README.md): +You do not need a token to run `npm install` unless your project is private. The token is only required to publish. If the `.npmrc` file was checked in with a reference to `$npm_TOKEN`, you can remove it. If you prefer to leave the reference in, you must set a value prior to running `npm install` or set the value by using [GitLab CI/CD variables](../../../ci/variables/index.md): ```shell NPM_TOKEN=<your_token> npm install @@ -532,12 +539,24 @@ If you get this error, ensure that: ### `npm publish` returns `npm ERR! 400 Bad Request` -If you get this error, your package name may not meet the +If you get this error, one of the following problems could be causing it. + +#### Package name does not meet the naming convention + +Your package name may not meet the [`@scope/package-name` package naming convention](#package-naming-convention). Ensure the name meets the convention exactly, including the case. Then try to publish again. +#### Package already exists + +Your package has already been published to another project in the same +root namespace and therefore cannot be published again using the same name. + +This is also true even if the prior published package shares the same name, +but not the version. + ### `npm publish` returns `npm ERR! 500 Internal Server Error - PUT` This is a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/238950) in GitLab diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index f19d565ef36..46cfd763668 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -8,6 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20050) in GitLab Premium 12.8. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Free in 13.3. +> - Symbol package support [added](https://gitlab.com/gitlab-org/gitlab/-/issues/262081) in GitLab 14.1. Publish NuGet packages in your project's Package Registry. Then, install the packages whenever you need to use them as a dependency. @@ -28,7 +29,7 @@ The required minimum versions are: - [NuGet CLI 5.1 or later](https://www.nuget.org/downloads). If you have [Visual Studio](https://visualstudio.microsoft.com/vs/), the NuGet CLI is probably already installed. -- Alternatively, you can use [.NET SDK 3.0 or later](https://dotnet.microsoft.com/download/dotnet-core/3.0), +- Alternatively, you can use [.NET SDK 3.0 or later](https://dotnet.microsoft.com/download/dotnet/3.0), which installs the NuGet CLI. - NuGet protocol version 3 or later. @@ -336,7 +337,7 @@ updated: stage: deploy script: - dotnet pack -c Release - - dotnet nuget add source "${CI_API_V4_URL}/${CI_PROJECT_ID}/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text + - dotnet nuget add source "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text - dotnet nuget push "bin/Release/*.nupkg" --source gitlab only: - master @@ -394,6 +395,24 @@ dotnet add package <package_id> \ - `<package_id>` is the package ID. - `<package_version>` is the package version. Optional. +## Symbol packages + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/262081) in GitLab 14.1. + +If you push a `.nupkg`, symbol package files in the `.snupkg` format are uploaded automatically. You +can also push them manually: + +```shell +nuget push My.Package.snupkg -Source <source_name> +``` + +Consuming symbol packages is not yet guaranteed using clients such as Visual Studio or +dotnet-symbol. The `.snupkg` files are available for download through the UI or the +[API](../../../api/packages/nuget.md#download-a-package-file). + +Follow the [NuGet symbol package issue](https://gitlab.com/gitlab-org/gitlab/-/issues/262081) +for further updates. + ## Supported CLI commands The GitLab NuGet repository supports the following commands for the NuGet CLI (`nuget`) and the .NET @@ -403,3 +422,12 @@ CLI (`dotnet`): - `dotnet nuget push`: Upload a package to the registry. - `nuget install`: Install a package from the registry. - `dotnet add`: Install a package from the registry. + +## Troubleshooting + +To improve performance, NuGet caches files related to a package. If you encounter issues, clear the +cache with this command: + +```shell +nuget locals all -clear +``` diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 52ce7d62940..cb4e677687e 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -34,8 +34,8 @@ For information on how to create and upload a package, view the GitLab documenta ## Use GitLab CI/CD to build packages -You can use [GitLab CI/CD](../../../ci/README.md) to build packages. -For Maven, NuGet, npm, Conan, and PyPI packages, and Composer dependencies, you can +You can use [GitLab CI/CD](../../../ci/index.md) to build packages. +For Maven, NuGet, npm, Conan, Helm, and PyPI packages, and Composer dependencies, you can authenticate with GitLab by using the `CI_JOB_TOKEN`. CI/CD templates, which you can use to get started, are in [this repository](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). @@ -94,7 +94,7 @@ To delete package files in the UI, from your group or project: 1. Go to **Packages & Registries > Package Registry**. 1. Find the name of the package you want to delete. 1. Select the package to view additional details. -1. Find the name of the file you would like to delete. +1. Find the name of the file you would like to delete. 1. Expand the ellipsis and select **Delete file**. The package files are permanently deleted. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 2dd00fdc273..30d61770094 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -204,7 +204,7 @@ Your project ID is on your project's home page. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. -To work with PyPI commands within [GitLab CI/CD](../../../ci/README.md), you +To work with PyPI commands within [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. For example: @@ -320,7 +320,7 @@ python -m twine upload --repository <source_name> dist/<package_file> You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first. If you attempt to publish the same package -more than once, a `404 Bad Request` error occurs. +more than once, a `400 Bad Request` error occurs. ## Install a PyPI package diff --git a/doc/user/packages/rubygems_registry/index.md b/doc/user/packages/rubygems_registry/index.md index 743bc229e11..2a0fea6e0ef 100644 --- a/doc/user/packages/rubygems_registry/index.md +++ b/doc/user/packages/rubygems_registry/index.md @@ -74,7 +74,7 @@ https://gitlab.example.com/api/v4/projects/<project_id>/packages/rubygems: '<you ### Authenticate with a CI job token -To work with RubyGems commands within [GitLab CI/CD](../../../ci/README.md), +To work with RubyGems commands within [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. For example: diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index efb2b8ddf8e..1365f401506 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -15,8 +15,8 @@ as a Terraform module registry. To authenticate to the Terraform module registry, you need either: -- A [personal access token](../../../api/README.md#personalproject-access-tokens) with at least `read_api` rights. -- A [CI/CD job token](../../../api/README.md#gitlab-cicd-job-token). +- A [personal access token](../../../api/index.md#personalproject-access-tokens) with at least `read_api` rights. +- A [CI/CD job token](../../../api/index.md#gitlab-cicd-job-token). ## Publish a Terraform Module @@ -26,7 +26,7 @@ If a package with the same name and version already exists, it will not be creat Prerequisites: -- You need to [authenticate with the API](../../../api/README.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. +- You need to [authenticate with the API](../../../api/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope. ```plaintext PUT /projects/:id/packages/terraform/modules/:module_name/:module_system/:module_version/file @@ -34,7 +34,7 @@ PUT /projects/:id/packages/terraform/modules/:module_name/:module_system/:module | Attribute | Type | Required | Description | | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/index.md#namespaced-path-encoding). | | `module_name` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), or hyphens (`-`). | `module_system` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), or hyphens (`-`). | `module_version` | string | yes | The package version. It must be valid according to the [Semantic Versioning Specification](https://semver.org/). @@ -77,7 +77,7 @@ Example response: Prerequisites: -- You need to [authenticate with the API](../../../api/README.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. +- You need to [authenticate with the API](../../../api/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: @@ -99,7 +99,7 @@ module "<module>" { ## Publish a Terraform module by using CI/CD -To work with Terraform modules in [GitLab CI/CD](../../../ci/README.md), you can use +To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. For example: diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 6739d08e156..0c3428ee7ee 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -49,8 +49,10 @@ The following table lists project permissions available for each role: | View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | -| View Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View License list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View Dependency list **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| View License list **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | +| View [Threats list](application_security/threat_monitoring/#threat-monitoring) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create and run [on-demand DAST scans](application_security/dast/#on-demand-scans) | | | ✓ | ✓ | ✓ | | View licenses in Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | | View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | @@ -59,7 +61,7 @@ The following table lists project permissions available for each role: | View wiki pages | ✓ | ✓ | ✓ | ✓ | ✓ | | See a list of jobs | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | See a job log | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | -| See a job with [debug logging](../ci/variables/README.md#debug-logging) | | | ✓ | ✓ | ✓ | +| See a job with [debug logging](../ci/variables/index.md#debug-logging) | | | ✓ | ✓ | ✓ | | Download and browse job artifacts | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | Create confidential issue | ✓ | ✓ | ✓ | ✓ | ✓ | | Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -105,8 +107,7 @@ The following table lists project permissions available for each role: | Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | | Create/edit/delete a Cleanup policy | | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | -| Create/edit [releases](project/releases/index.md)| | | ✓ | ✓ | ✓ | -| Delete [releases](project/releases/index.md)| | | | ✓ | ✓ | +| Create/edit/delete [releases](project/releases/index.md)| | | ✓ (*13*) | ✓ (*13*) | ✓ (*13*) | | Manage merge approval rules (project settings) | | | | ✓ | ✓ | | Create new merge request | | | ✓ | ✓ | ✓ | | Create new branches | | | ✓ | ✓ | ✓ | @@ -169,6 +170,8 @@ The following table lists project permissions available for each role: | Manage Project Operations | | | | ✓ | ✓ | | Manage Terraform state | | | | ✓ | ✓ | | Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | +| Manage security policy **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create or assign security policy project **(ULTIMATE)** | | | | | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | | Reposition comments on images (posted by any user)|✓ (*10*) | ✓ (*10*) | ✓ (*10*) | ✓ | ✓ | | Manage Error Tracking | | | | ✓ | ✓ | @@ -195,7 +198,7 @@ The following table lists project permissions available for each role: 1. Guest users can only view the confidential issues they created themselves. 1. If **Public pipelines** is enabled in **Project Settings > CI/CD**. 1. Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [protected branches](project/protected_branches.md). -1. If the [branch is protected](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings), this depends on the access Developers and Maintainers are given. +1. If the [branch is protected](project/protected_branches.md), this depends on the access Developers and Maintainers are given. 1. Guest users can access GitLab [**Releases**](project/releases/index.md) for downloading assets but are not allowed to download the source code nor see repository information like tags and commits. 1. Actions are limited only to records owned (referenced) by user. 1. When [Share Group Lock](group/index.md#prevent-a-project-from-being-shared-with-groups) is enabled the project can't be shared with other groups. It does not affect group with group sharing. @@ -204,7 +207,8 @@ The following table lists project permissions available for each role: 1. Applies only to comments on [Design Management](project/issues/design_management.md) designs. 1. Users can only view events based on their individual actions. 1. Project access tokens are supported for self-managed instances on Free and above. They are also - supported on GitLab SaaS Premium and above (excluding [trial licenses](https://about.gitlab.com/free-trial)). + supported on GitLab SaaS Premium and above (excluding [trial licenses](https://about.gitlab.com/free-trial/)). +1. If the [tag is protected](#release-permissions-with-protected-tags), this depends on the access Developers and Maintainers are given. ## Project features permissions @@ -223,7 +227,7 @@ which visibility level you select on project settings. Additional restrictions can be applied on a per-branch basis with [protected branches](project/protected_branches.md). Additionally, you can customize permissions to allow or prevent project Maintainers and Developers from pushing to a protected branch. Read through the documentation on -[Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) +[protected branches](project/protected_branches.md) to learn more. ### Value Stream Analytics permissions @@ -261,50 +265,50 @@ The following table lists group permissions available for each role: | Action | Guest | Reporter | Developer | Maintainer | Owner | |--------------------------------------------------------|-------|----------|-----------|------------|-------| | Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| Edit SAML SSO Billing **(PREMIUM SAAS)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | +| View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| View group epic **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View group wiki pages **(PREMIUM)** | ✓ (6) | ✓ | ✓ | ✓ | ✓ | +| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View Insights charts **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View group epic **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | | Create/edit group epic **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | Create/edit/delete epic boards **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | -| See a container registry | | ✓ | ✓ | ✓ | ✓ | | Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | -| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | +| View a container registry | | ✓ | ✓ | ✓ | ✓ | +| View Group DevOps Adoption **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | +| View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | +| Create and edit group wiki pages **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | -| Share (invite) groups with groups | | | | | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | | Create/edit/delete iterations | | | ✓ | ✓ | ✓ | +| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy | | | ✓ | ✓ | ✓ | -| Create and edit group wiki pages **(PREMIUM)** | | | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | -| View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | +| View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | | Create subgroup | | | | ✓ (1) | ✓ | | Delete group wiki pages **(PREMIUM)** | | | | ✓ | ✓ | | Edit epic comments (posted by any user) **(ULTIMATE)** | | | | ✓ (2) | ✓ (2) | -| Edit group settings | | | | | ✓ | -| Manage group level CI/CD variables | | | | | ✓ | | List group deploy tokens | | | | ✓ | ✓ | +| Manage [group push rules](group/index.md#group-push-rules) **(PREMIUM)** | | | | ✓ | ✓ | +| View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | +| Administer project compliance frameworks | | | | | ✓ | | Create/Delete group deploy tokens | | | | | ✓ | -| Manage group members | | | | | ✓ | | Delete group | | | | | ✓ | | Delete group epic **(PREMIUM)** | | | | | ✓ | -| Edit SAML SSO Billing **(PREMIUM SAAS)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | -| View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | | Disable notification emails | | | | | ✓ | -| View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Group DevOps Adoption **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| Edit group settings | | | | | ✓ | +| Filter members by 2FA status | | | | | ✓ | +| Manage group level CI/CD variables | | | | | ✓ | +| Manage group members | | | | | ✓ | +| Share (invite) groups with groups | | | | | ✓ | +| View 2FA status of members | | | | | ✓ | | View Billing **(FREE SAAS)** | | | | | ✓ (4) | | View Usage Quotas **(FREE SAAS)** | | | | | ✓ (4) | -| Manage [group push rules](group/index.md#group-push-rules) **(PREMIUM)** | | | | ✓ | ✓ | -| View 2FA status of members | | | | | ✓ | -| Filter members by 2FA status | | | | | ✓ | -| Administer project compliance frameworks | | | | | ✓ | 1. Groups can be set to [allow either Owners or Owners and Maintainers to create subgroups](group/subgroups/index.md#creating-a-subgroup) @@ -521,6 +525,14 @@ run CI/CD pipelines and execute actions on jobs that are related to those branch See [Security on protected branches](../ci/pipelines/index.md#pipeline-security-on-protected-branches) for details about the pipelines security model. +## Release permissions with protected tags + +[The permission to create tags](project/protected_tags.md) is used to define if a user can +create, edit, and delete [Releases](project/releases/index.md). + +See [Release permissions](project/releases/index.md#release-permissions) +for more information. + ## LDAP users permissions In GitLab 8.15 and later, LDAP user permissions can now be manually overridden by an admin user. diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index 3f42fc0d131..597170540ab 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -366,7 +366,7 @@ Support for disabling 2FA is limited, depending on your subscription level. For When 2FA is enabled, you can no longer use your normal account password to authenticate with Git over HTTPS on the command line or when using -the [GitLab API](../../../api/README.md). You must use a +the [GitLab API](../../../api/index.md). You must use a [personal access token](../personal_access_tokens.md) instead. ## Recovery options @@ -507,7 +507,7 @@ If you are receiving an `invalid pin code` error, this may indicate that there i To avoid the time sync issue, enable time synchronization in the device that generates the codes. For example: -- For Android (Google Authenticator): +- For Android (Google Authenticator): 1. Go to the Main Menu in Google Authenticator. 1. Select Settings. 1. Select the Time correction for the codes. diff --git a/doc/user/profile/img/unknown_sign_in_email_v14_0.png b/doc/user/profile/img/unknown_sign_in_email_v14_0.png Binary files differindex dec1251addb..62634739b78 100644 --- a/doc/user/profile/img/unknown_sign_in_email_v14_0.png +++ b/doc/user/profile/img/unknown_sign_in_email_v14_0.png diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 9d714a6efd0..d49f4ab0c16 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -79,6 +79,16 @@ The following is hidden from your user profile page (`https://gitlab.example.com NOTE: Making your user profile page private does not hide your public resources from the REST or GraphQL APIs. +### User visibility + +The public page of a user, located at `/username`, is always visible whether you are signed-in or +not. + +When visiting the public page of a user, you can only see the projects which you have privileges to. + +If the [public level is restricted](../admin_area/settings/visibility_and_access_controls.md#restricted-visibility-levels), +user profiles are only visible to signed-in users. + ## Add external accounts to your user profile page You can add links to certain other external accounts you might have, like Skype and Twitter. @@ -98,7 +108,7 @@ To add links to other accounts: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/14078) in GitLab 11.3. -In the user contribution calendar graph and recent activity list, you can show contributions to private projects. +In the user contribution calendar graph and recent activity list, you can see your [contribution actions](../../api/events.md#action-types) to private projects. To show private contributions: @@ -280,6 +290,6 @@ Without the `config.extend_remember_period` flag, you would be forced to sign in - [Receive emails for sign-ins from unknown IP addresses or devices](unknown_sign_in_notification.md) - Manage applications that can [use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth) - Manage [personal access tokens](personal_access_tokens.md) to access your account via API and authorized applications -- Manage [SSH keys](../../ssh/README.md) to access your account via SSH +- Manage [SSH keys](../../ssh/index.md) to access your account via SSH - Change your [syntax highlighting theme](preferences.md#syntax-highlighting-theme) - [View your active sessions](active_sessions.md) and revoke any of them if necessary diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index b9410be791e..eaf1d33a938 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -176,14 +176,14 @@ Users are notified of the following events: | Event | Sent to | Settings level | |------------------------------|---------------------|------------------------------| | New SSH key added | User | Security email, always sent. | -| SSH key has expired | User | Security email, always sent. | +| SSH key has expired | User | Security email, always sent. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.12 | | New email added | User | Security email, always sent. | | Email changed | User | Security email, always sent. | | Password changed | User | Security email, always sent when user changes their own password | | Password changed by administrator | User | Security email, always sent when an administrator changes the password of another user | | Two-factor authentication disabled | User | Security email, always sent. | | New user created | User | Sent on user creation, except for OmniAuth (LDAP)| -| New SAML/SCIM user provisioned. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276018) in GitLab 13.8 | User | Sent when a user is provisioned through SAML/SCIM | +| New SAML/SCIM user provisioned | User | Sent when a user is provisioned through SAML/SCIM. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276018) in GitLab 13.8 | | User added to project | User | Sent when user is added to project | | Project access level changed | User | Sent when user project access level is changed | | User added to group | User | Sent when user is added to group | @@ -237,10 +237,10 @@ epics: | Close merge request | | | Due issue | Participants and Custom notification level with this event selected | | Failed pipeline | The author of the pipeline | -| Fixed pipeline ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24309) in GitLab 13.1.) | The author of the pipeline. Enabled by default. | +| Fixed pipeline | The author of the pipeline. Enabled by default. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24309) in GitLab 13.1 | | Merge merge request | | -| Merge when pipeline succeeds ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211961) in GitLab 13.4) | Author, Participants, Watchers, Subscribers, and Custom notification level with this event selected. `Note:` Custom notification level is ignored for Author, Watchers and Subscribers | -| Merge request [marked as ready](../project/merge_requests/drafts.md) (introduced in [GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/15332)) | Watchers and participants | +| Merge when pipeline succeeds | Author, Participants, Watchers, Subscribers, and Custom notification level with this event selected. Custom notification level is ignored for Author, Watchers and Subscribers. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211961) in GitLab 13.4 | +| Merge request [marked as ready](../project/merge_requests/drafts.md) | Watchers and participants. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15332) in GitLab 13.10 | | New comment | Participants, Watchers, Subscribers, and Custom notification level with this event selected, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher | | New epic | | | New issue | | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 7b63a5bfef9..0dbf00a83fb 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -11,16 +11,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Notifications for expiring tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) added in GitLab 12.6. > - [Token lifetime limits](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) added in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. > - [Additional notifications for expiring tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/214721) added in GitLab 13.3. +> - [Prefill token name and scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/334664) added in GitLab 14.1. -If you're unable to use [OAuth2](../../api/oauth2.md), you can use a personal access token to authenticate with the [GitLab API](../../api/README.md#personalproject-access-tokens). You can also use a personal access token with Git to authenticate over HTTP. +If you're unable to use [OAuth2](../../api/oauth2.md), you can use a personal access token to authenticate with the [GitLab API](../../api/index.md#personalproject-access-tokens). You can also use a personal access token with Git to authenticate over HTTP. In both cases, you authenticate with a personal access token in place of your password. -Personal access tokens are required when [Two-Factor Authentication (2FA)](account/two_factor_authentication.md) is enabled. +Personal access tokens are required when [Two-Factor Authentication (2FA)](account/two_factor_authentication.md) is enabled. -For examples of how you can use a personal access token to authenticate with the API, see the [API documentation](../../api/README.md#personalproject-access-tokens). +For examples of how you can use a personal access token to authenticate with the API, see the [API documentation](../../api/index.md#personalproject-access-tokens). -Alternately, GitLab administrators can use the API to create [impersonation tokens](../../api/README.md#impersonation-tokens). +Alternately, GitLab administrators can use the API to create [impersonation tokens](../../api/index.md#impersonation-tokens). Use impersonation tokens to automate authentication as a specific user. ## Create a personal access token @@ -37,6 +38,16 @@ You can create as many personal access tokens as you like. Save the personal access token somewhere safe. After you leave the page, you no longer have access to the token. +### Prefill personal access token name and scopes + +You can link directly to the Personal Access Token page and have the form prefilled with a name and +list of scopes. To do this, you can append a `name` parameter and a list of comma-separated scopes +to the URL. For example: + +```plaintext +https://gitlab.example.com/-/profile/personal_access_tokens?name=Example+Access+token&scopes=api,read_user,read_registry +``` + ## Revoke a personal access token At any time, you can revoke a personal access token. @@ -82,7 +93,7 @@ Personal access tokens expire on the date you define, at midnight UTC. - In GitLab Ultimate, administrators can [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens). - In GitLab Ultimate, administrators can choose whether or not to - [enforce personal access token expiration](../admin_area/settings/account_and_limit_settings.md#do-not-enforce-personal-access-token-expiration). + [enforce personal access token expiration](../admin_area/settings/account_and_limit_settings.md#allow-expired-personal-access-tokens-to-be-used). ## Create a personal access token programmatically **(FREE SELF)** @@ -104,10 +115,10 @@ To create a personal access token programmatically: ``` 1. Run the following commands to reference the username, the token, and the scopes. - + The token must be 20 characters long. The scopes must be valid and are visible [in the source code](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/auth.rb). - + For example, to create a token that belongs to a user with username `automation-bot`: ```ruby @@ -141,7 +152,7 @@ To revoke a token programmatically: ```shell sudo gitlab-rails console ``` - + 1. To revoke a token of `token-string-here123`, run the following commands: ```ruby diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 540724f9185..fad9b67eee4 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -163,7 +163,31 @@ You can choose one of the following options as the first day of the week: - Sunday - Monday -If you select **System Default**, the system-wide default setting is used. +If you select **System Default**, the [instance default](../admin_area/settings/index.md#default-first-day-of-the-week) setting is used. + +## Time preferences + +### Use relative times + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/65570) in GitLab 14.1. + +You can select your preferred time format for the GitLab user interface: + +- Relative times, for example, `30 minutes ago`. +- Absolute times, for example, `May 18, 2021, 3:57 PM`. + +The times are formatted depending on your chosen language and browser locale. + +To set your time preference: + +1. On the **Preferences** page, go to **Time preferences**. +1. Select the **Use relative times** checkbox to use relative times, + or clear the checkbox to use absolute times. +1. Select **Save changes**. + +NOTE: +This feature is experimental, and choosing absolute times might break certain layouts. +Please open an issue if you notice that using absolute times breaks a layout. ## Integrations diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index c3900d33cb8..cac283f6f18 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -92,7 +92,7 @@ Here's an example setup flow from scratch: 1. Prepare an [Auto DevOps-enabled](../../topics/autodevops/index.md) project. 1. Set up a [Kubernetes Cluster](../../user/project/clusters/index.md) in your project. 1. Install [NGINX Ingress](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx) in your cluster. -1. Set up [the base domain](../../user/project/clusters/index.md#base-domain) based on the Ingress +1. Set up [the base domain](../../user/project/clusters/gitlab_managed_clusters.md#base-domain) based on the Ingress Endpoint assigned above. 1. Check if [`v2.0.0+` of `auto-deploy-image` is used in your Auto DevOps pipelines](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#verify-dependency-versions). If it isn't, follow the documentation to specify the image version. diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 58bdb3d698f..7d006247177 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -4,85 +4,57 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Adding EKS clusters **(FREE)** +# EKS clusters (DEPRECATED) **(FREE)** -GitLab supports adding new and existing EKS clusters. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22392) in GitLab 12.5. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -## EKS requirements +WARNING: +Use [Infrastrucure as Code](../../infrastructure/index.md) to create new clusters. The method described in this document is deprecated as of GitLab 14.0. -Before creating your first cluster on Amazon EKS with the GitLab integration, make sure the following -requirements are met: +Through GitLab, you can create new clusters and add existing clusters hosted on Amazon Elastic +Kubernetes Service (EKS). -- An [Amazon Web Services](https://aws.amazon.com/) account is set up and you are able to log in. -- You have permissions to manage IAM resources. -- If you want to use an [existing EKS cluster](#existing-eks-cluster): - - An Amazon EKS cluster with worker nodes properly configured. - - `kubectl` [installed and configured](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) - for access to the EKS cluster. +## Add an existing EKS cluster -### Additional requirements for self-managed instances **(FREE SELF)** +If you already have an EKS cluster and want to integrate it with GitLab, +see how to [add an existing cluster](add_existing_cluster.md). -If you are using a self-managed GitLab instance, GitLab must first be configured with a set of -Amazon credentials. These credentials are used to assume an Amazon IAM role provided by the user -creating the cluster. Create an IAM user and ensure it has permissions to assume the role(s) that -your users need to create EKS clusters. +## Create a new certificate-based EKS cluster -For example, the following policy document allows assuming a role whose name starts with -`gitlab-eks-` in account `123456789012`: +Prerequisites: -```json -{ - "Version": "2012-10-17", - "Statement": { - "Effect": "Allow", - "Action": "sts:AssumeRole", - "Resource": "arn:aws:iam::123456789012:role/gitlab-eks-*" - } -} -``` +- An [Amazon Web Services](https://aws.amazon.com/) account. +- Permissions to manage IAM resources. -### Configure Amazon authentication +For instance-level clusters, see [additional requirements for self-managed instances](#additional-requirements-for-self-managed-instances). **(FREE SELF)** -To configure Amazon authentication in GitLab, generate an access key for the IAM user in the Amazon AWS console, and following the steps below. +To create new Kubernetes clusters for your project, group, or instance through the certificate-based method: -1. Navigate to **Admin Area > Settings > General** and expand the **Amazon EKS** section. -1. Check **Enable Amazon EKS integration**. -1. Enter your **Account ID**. -1. Depending on your configuration, enter your access key and ID: +1. [Define the access control (RBAC or ABAC) for your cluster](cluster_access.md). +1. [Create a cluster in GitLab](#create-a-new-eks-cluster-in-gitlab). +1. [Prepare the cluster in Amazon](#prepare-the-cluster-in-amazon). +1. [Configure your cluster's data in GitLab](#configure-your-clusters-data-in-gitlab). - - _GitLab 13.7 and later, and using an instance profile_: You may leave - **Access key ID** and **Secret access key** blank. - Read [Instance profiles](#instance-profiles) for more information. - - _All GitLab versions_: Enter your access key credentials into - **Access key ID** and **Secret access key**. +Further steps: -1. Click **Save changes**. +1. [Create a default Storage Class](#create-a-default-storage-class). +1. [Deploy the app to EKS](#deploy-the-app-to-eks). -#### Instance profiles +### Create a new EKS cluster in GitLab -> Introduced in [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/issues/291015). +To create a new EKS cluster: -You may leave `Access key ID` and `Secret access key` fields blank if -you are using an instance profile -[to pass an IAM role to an EC2 instance](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html). -Instance profiles dynamically retrieve temporary credentials from AWS when needed. - -## New EKS cluster - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22392) in GitLab 12.5. - -To create and add a new Kubernetes cluster to your project, group, or instance: - -1. Navigate to your: +1. Go to your: - Project's **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **Kubernetes** page, for a group-level cluster. - - **Admin Area > Kubernetes**, for an instance-level cluster. -1. Click **Integrate with a cluster certificate**. + - **Menu >** **{admin}** **Admin > Kubernetes**, for an instance-level cluster. +1. Select **Integrate with a cluster certificate**. 1. Under the **Create new cluster** tab, click **Amazon EKS** to display an `Account ID` and `External ID` needed for later steps. 1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an IAM policy: 1. From the left panel, select **Policies**. - 1. Click **Create Policy**, which opens a new window. + 1. Select **Create Policy**, which opens a new window. 1. Select the **JSON** tab, and paste the following snippet in place of the existing content. These permissions give GitLab the ability to create resources, but not delete them: @@ -133,132 +105,163 @@ To create and add a new Kubernetes cluster to your project, group, or instance: } ``` - If an error is encountered during the creation process, changes will - not be rolled back and you must remove resources manually. You can do this by deleting - the relevant [CloudFormation stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-delete-stack.html) + If you get an error during this process, GitLab does not roll back the changes. You must remove resources manually. You can do this by deleting + the relevant [CloudFormation stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-delete-stack.html). 1. Click **Review policy**. 1. Enter a suitable name for this policy, and click **Create Policy**. You can now close this window. -1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an **EKS IAM role** following the [Amazon EKS cluster IAM role instructions](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html). This role should exist so that Kubernetes clusters managed by Amazon EKS can make calls to other AWS services on your behalf to manage the resources that you use with the service. - In addition to the policies that guide suggests, you must also include the `AmazonEKSClusterPolicy` - policy for this role in order for GitLab to manage the EKS cluster correctly. -1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create another IAM role which will be used by GitLab to authenticate with AWS. Follow these steps to create it: - 1. On the AWS IAM console, select **Roles** from the left panel. - 1. Click **Create role**. - 1. Under `Select type of trusted entity`, select **Another AWS account**. - 1. Enter the Account ID from GitLab into the `Account ID` field. - 1. Check **Require external ID**. - 1. Enter the External ID from GitLab into the `External ID` field. - 1. Click **Next: Permissions**, and select the policy you just created. - 1. Click **Next: Tags**, and optionally enter any tags you wish to associate with this role. - 1. Click **Next: Review**. - 1. Enter a role name and optional description into the fields provided. - 1. Click **Create role**, the new role name displays at the top. Click on its name and copy the `Role ARN` from the newly created role. -1. In GitLab, enter the copied role ARN into the `Role ARN` field. +### Prepare the cluster in Amazon + +1. [Create an **EKS IAM role** for your cluster](#create-an-eks-iam-role-for-your-cluster) (**role A**). +1. [Create **another EKS IAM role** for GitLab authentication with Amazon](#create-another-eks-iam-role-for-gitlab-authentication-with-amazon) (**role B**). + +#### Create an EKS IAM role for your cluster + +In the [IAM Management Console](https://console.aws.amazon.com/iam/home), +create an **EKS IAM role** (**role A**) following the [Amazon EKS cluster IAM role instructions](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html). +This role is necessary so that Kubernetes clusters managed by Amazon EKS can make calls to other AWS +services on your behalf to manage the resources that you use with the service. + +For GitLab to manage the EKS cluster correctly, you must include `AmazonEKSClusterPolicy` in +addition to the policies the guide suggests. + +#### Create another EKS IAM role for GitLab authentication with Amazon + +In the [IAM Management Console](https://console.aws.amazon.com/iam/home), +create another IAM role (**role B**) for GitLab authentication with AWS: + +1. On the AWS IAM console, select **Roles** from the left panel. +1. Click **Create role**. +1. Under **Select type of trusted entity**, select **Another AWS account**. +1. Enter the Account ID from GitLab into the **Account ID** field. +1. Check **Require external ID**. +1. Enter the External ID from GitLab into the **External ID** field. +1. Click **Next: Permissions**, and select the policy you just created. +1. Click **Next: Tags**, and optionally enter any tags you wish to associate with this role. +1. Click **Next: Review**. +1. Enter a role name and optional description into the fields provided. +1. Click **Create role**. The new role name displays at the top. Click on its name and copy the + `Role ARN` from the newly created role. + +### Configure your cluster's data in GitLab + +1. Back in GitLab, enter the copied role ARN into the **Role ARN** field. 1. In the **Cluster Region** field, enter the [region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html) you plan to use for your new cluster. GitLab confirms you have access to this region when authenticating your role. -1. Click **Authenticate with AWS**. -1. Choose your cluster's settings: - - **Kubernetes cluster name** - The name you wish to give the cluster. - - **Environment scope** - The [associated environment](index.md#setting-the-environment-scope) to this cluster. - - **Kubernetes version** - The [Kubernetes version](index.md#supported-cluster-versions) to use. - - **Service role** - Select the **EKS IAM role** you created earlier to allow Amazon EKS - and the Kubernetes control plane to manage AWS resources on your behalf. - - NOTE: - This IAM role is _not_ the IAM role you created in the previous step. It should be - the one you created much earlier by following the - [Amazon EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html) - guide. - - **Key pair name** - Select the [key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html) - that you can use to connect to your worker nodes if required. - - **VPC** - Select a [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) - to use for your EKS Cluster resources. - - **Subnets** - Choose the [subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) - in your VPC where your worker nodes run. You must select at least two. - - **Security group** - Choose the [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) - to apply to the EKS-managed Elastic Network Interfaces that are created in your worker node subnets. - - **Instance type** - The [instance type](https://aws.amazon.com/ec2/instance-types/) of your worker nodes. - - **Node count** - The number of worker nodes. - - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. - See the [Managed clusters section](index.md#gitlab-managed-clusters) for more information. -1. Finally, click the **Create Kubernetes cluster** button. +1. Select **Authenticate with AWS**. +1. Adjust your [cluster's settings](#cluster-settings). +1. Select the **Create Kubernetes cluster** button. After about 10 minutes, your cluster is ready to go. NOTE: -If you have [installed and configured](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) `kubectl` and you would like to manage your cluster with it, you must add your AWS external ID in the AWS configuration. For more information on how to configure AWS CLI, see [using an IAM role in the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html#cli-configure-role-xaccount). - -### Cluster creation flow - -The following sequence illustrates how GitLab works with AWS to create an EKS cluster: - -```mermaid -sequenceDiagram - autonumber - participant G as GitLab - participant A as AWS - participant E as EKS cluster - alt static credentials - G->>G: Load AWS Access and secret key - end - alt IAM instance profile - G->>A: Fetch temporary credentials - A->>G: Temporary access credentials - end - G->>A: AssumeRole: EKS Provision Role - A->>A: Check account, external IDs - A->>A: Check permissions - A->>G: New access credentials - note over G: user selects EKS cluster options - note over G,A: Use Service Role credentials - G->>A: CreateStack (CloudFormation) - A->>G: Received - G->>G: Wait 5 minutes - loop Poll for cluster creation - G->>A: DescribeStacks - A->>G: CREATE_IN_PROGRESS - end - note over G,E: EKS Cluster Created - G->>A: DescribeStacks - A->>G: CREATE_COMPLETE - G->>E: kubectl create role (service account) - E->>G: OK +If you have [installed and configured](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) `kubectl` and you would like to manage your cluster with it, you must add your AWS external ID in the AWS configuration. For more information on how to configure AWS CLI, see [using an IAM role in the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html#cli-configure-role-xaccount). + +#### Cluster settings + +When you create a new cluster, you have the following settings: + +| Setting | Description | +| ----------------------- |------------ | +| Kubernetes cluster name | Your cluster's name. | +| Environment scope | The [associated environment](multiple_kubernetes_clusters.md#setting-the-environment-scope). | +| Service role | The **EKS IAM role** (**role A**). | +| Kubernetes version | The [Kubernetes version](index.md#supported-cluster-versions) for your cluster. | +| Key pair name | The [key pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html) that you can use to connect to your worker nodes. | +| VPC | The [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) to use for your EKS Cluster resources. | +| Subnets | The [subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) in your VPC where your worker nodes run. Two are required. | +| Security group | The [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) to apply to the EKS-managed Elastic Network Interfaces that are created in your worker node subnets. | +| Instance type | The [instance type](https://aws.amazon.com/ec2/instance-types/) of your worker nodes. | +| Node count | The number of worker nodes. | +| GitLab-managed cluster | Check if you want GitLab to manage namespaces and service accounts for this cluster. | + +## Create a default Storage Class + +Amazon EKS doesn't have a default Storage Class out of the box, which means +requests for persistent volumes are not automatically fulfilled. As part +of Auto DevOps, the deployed PostgreSQL instance requests persistent storage, +and without a default storage class it cannot start. + +If a default Storage Class doesn't already exist and is desired, follow Amazon's +[guide on storage classes](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html) +to create one. + +Alternatively, disable PostgreSQL by setting the project variable +[`POSTGRES_ENABLED`](../../../topics/autodevops/customize.md#cicd-variables) to `false`. + +## Deploy the app to EKS + +With RBAC disabled and services deployed, +[Auto DevOps](../../../topics/autodevops/index.md) can now be leveraged +to build, test, and deploy the app. + +[Enable Auto DevOps](../../../topics/autodevops/index.md#at-the-project-level) +if not already enabled. If a wildcard DNS entry was created resolving to the +Load Balancer, enter it in the `domain` field under the Auto DevOps settings. +Otherwise, the deployed app isn't externally available outside of the cluster. + + + +GitLab creates a new pipeline, which begins to build, test, and deploy the app. + +After the pipeline has finished, your app runs in EKS, and is available +to users. Click on **CI/CD > Environments**. + + + +GitLab displays a list of the environments and their deploy status, as well as +options to browse to the app, view monitoring metrics, and even access a shell +on the running pod. + +## Additional requirements for self-managed instances **(FREE SELF)** + +If you are using a self-managed GitLab instance, you need to configure +Amazon credentials. GitLab uses these credentials to assume an Amazon IAM role to create your cluster. + +Create an IAM user and ensure it has permissions to assume the role(s) that +your users need to create EKS clusters. + +For example, the following policy document allows assuming a role whose name starts with +`gitlab-eks-` in account `123456789012`: + +```json +{ + "Version": "2012-10-17", + "Statement": { + "Effect": "Allow", + "Action": "sts:AssumeRole", + "Resource": "arn:aws:iam::123456789012:role/gitlab-eks-*" + } +} ``` -First, GitLab must obtain an initial set of credentials to communicate with the AWS API. -These credentials can be retrieved in one of two ways: +### Configure Amazon authentication + +To configure Amazon authentication in GitLab, generate an access key for the +IAM user in the Amazon AWS console, and follow these steps: -- Statically through the [Configure Amazon authentication](#configure-amazon-authentication). -- Dynamically via an IAM instance profile ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291015) in GitLab 13.7). +1. In GitLab, on the top bar, select **Menu >** **{admin}** **Admin > Settings > General** and expand the **Amazon EKS** section. +1. Check **Enable Amazon EKS integration**. +1. Enter your **Account ID**. +1. Enter your [access key and ID](#eks-access-key-and-id). +1. Click **Save changes**. -After GitLab retrieves the AWS credentials, it makes an -[AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) -API call to obtain credentials for the Provision Role. AWS confirms -the request has the correct account ID, external ID, and permissions. +#### EKS access key and ID -If the request is valid, AWS returns a new set of temporary credentials GitLab -uses to load the **Create cluster** options page. +> Instance profiles were [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291015) in GitLab 13.7. -On the **Create cluster** page, the user must select a **Service Role**, which is -the IAM role that is actually used to create the cluster, and other options -such as the Kubernetes cluster name, Kubernetes version, and region. -After the user clicks the **Create Kubernetes cluster** button, GitLab -submits a CloudFormation API request to create an EKS cluster with the given parameters -from the user. GitLab waits 5 minutes before checking whether the cluster was created, -and polls once a minute for up to 30 minutes. +If you're using GitLab 13.7 or later, you can use instance profiles to +dynamically retrieve temporary credentials from AWS when needed. +In this case, leave the `Access key ID` and `Secret access key` fields blank +and [pass an IAM role to an EC2 instance](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html). -After GitLab receives a `CREATE_COMPLETE` message from AWS, GitLab talks -to the EKS cluster to create a Kubernetes service account with `cluster-admin` -privileges, and updates its internal database to reflect the newly-created -Kubernetes cluster. From this point forward, GitLab uses this service account to -interact with the cluster. +Otherwise, enter your access key credentials into **Access key ID** and **Secret access key**. -### Troubleshooting creating a new cluster +## Troubleshooting The following errors are commonly encountered when creating a new cluster. -#### Validation failed: Role ARN must be a valid Amazon Resource Name +### Validation failed: Role ARN must be a valid Amazon Resource Name Check that the `Provision Role ARN` is correct. An example of a valid ARN: @@ -266,7 +269,7 @@ Check that the `Provision Role ARN` is correct. An example of a valid ARN: arn:aws:iam::123456789012:role/gitlab-eks-provision' ``` -#### Access denied: User `arn:aws:iam::x` is not authorized to perform: `sts:AssumeRole` on resource: `arn:aws:iam::y` +### Access denied: User `arn:aws:iam::x` is not authorized to perform: `sts:AssumeRole` on resource: `arn:aws:iam::y` This error occurs when the credentials defined in the [Configure Amazon authentication](#configure-amazon-authentication) cannot assume the role defined by the @@ -280,7 +283,7 @@ Provision Role ARN. Check that:  -#### Could not load Security Groups for this VPC +### Could not load Security Groups for this VPC When populating options in the configuration form, GitLab returns this error because GitLab has successfully assumed your provided role, but the role has @@ -307,46 +310,3 @@ This role should be the role you created by following the [EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html) guide. In addition to the policies that guide suggests, you must also include the `AmazonEKSClusterPolicy` policy for this role in order for GitLab to manage the EKS cluster correctly. - -## Existing EKS cluster - -For information on adding an existing EKS cluster, see -[Existing Kubernetes cluster](add_remove_clusters.md#existing-kubernetes-cluster). - -### Create a default Storage Class - -Amazon EKS doesn't have a default Storage Class out of the box, which means -requests for persistent volumes are not automatically fulfilled. As part -of Auto DevOps, the deployed PostgreSQL instance requests persistent storage, -and without a default storage class it cannot start. - -If a default Storage Class doesn't already exist and is desired, follow Amazon's -[guide on storage classes](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html) -to create one. - -Alternatively, disable PostgreSQL by setting the project variable -[`POSTGRES_ENABLED`](../../../topics/autodevops/customize.md#cicd-variables) to `false`. - -### Deploy the app to EKS - -With RBAC disabled and services deployed, -[Auto DevOps](../../../topics/autodevops/index.md) can now be leveraged -to build, test, and deploy the app. - -[Enable Auto DevOps](../../../topics/autodevops/index.md#at-the-project-level) -if not already enabled. If a wildcard DNS entry was created resolving to the -Load Balancer, enter it in the `domain` field under the Auto DevOps settings. -Otherwise, the deployed app isn't externally available outside of the cluster. - - - -GitLab creates a new pipeline, which begins to build, test, and deploy the app. - -After the pipeline has finished, your app runs in EKS, and is available -to users. Click on **CI/CD > Environments**. - - - -GitLab displays a list of the environments and their deploy status, as well as -options to browse to the app, view monitoring metrics, and even access a shell -on the running pod. diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md new file mode 100644 index 00000000000..efd480fa3ce --- /dev/null +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -0,0 +1,224 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Add an existing Kubernetes cluster + +If you have an existing Kubernetes cluster, you can add it to a project, group, +or instance and benefit from the integration with GitLab. + +## Prerequisites + +See the prerequisites below to add existing clusters to GitLab. + +### All clusters + +To add any cluster to GitLab, you need: + +- Either a GitLab.com account or an account for a self-managed installation +running GitLab 12.5 or later. +- The Maintainer role for group-level and project-level clusters. +- Access to the Admin area for instance-level clusters. **(FREE SELF)** +- A Kubernetes cluster. +- Cluster administration access to the cluster with `kubectl`. + +You can host your cluster in [EKS](#eks-clusters), [GKE](#gke-clusters), +on premises, and with other providers. +To host them on premises and with other providers, +use either the EKS or GKE method to guide you through and enter your cluster's +settings manually. + +WARNING: +GitLab doesn't support `arm64` clusters. See the issue +[Helm Tiller fails to install on `arm64` cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838) +for details. + +### EKS clusters + +To add an existing **EKS** cluster, you need: + +- An Amazon EKS cluster with worker nodes properly configured. +- `kubectl` [installed and configured](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) +for access to the EKS cluster. +- Ensure the token of the account has administrator privileges for the cluster. + +### GKE clusters + +To add an existing **GKE** cluster, you need: + +- The `container.clusterRoleBindings.create` permission to create a cluster +role binding. You can follow the [Google Cloud documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access) +to grant access. + +## How to add an existing cluster + +<!-- (REVISE - BREAK INTO SMALLER STEPS) --> + +To add a Kubernetes cluster to your project, group, or instance: + +1. Navigate to your: + 1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. + 1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. + 1. **Menu >** **{admin}** **Admin >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Click **Add Kubernetes cluster**. +1. Click the **Add existing cluster** tab and fill in the details: + 1. **Kubernetes cluster name** (required) - The name you wish to give the cluster. + 1. **Environment scope** (required) - The + [associated environment](multiple_kubernetes_clusters.md#setting-the-environment-scope) to this cluster. + 1. **API URL** (required) - + It's the URL that GitLab uses to access the Kubernetes API. Kubernetes + exposes several APIs, we want the "base" URL that is common to all of them. + For example, `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. + + Get the API URL by running this command: + + ```shell + kubectl cluster-info | grep -E 'Kubernetes master|Kubernetes control plane' | awk '/http/ {print $NF}' + ``` + + 1. **CA certificate** (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We use the certificate created by default. + 1. List the secrets with `kubectl get secrets`, and one should be named similar to + `default-token-xxxxx`. Copy that token name for use below. + 1. Get the certificate by running this command: + + ```shell + kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode + ``` + + If the command returns the entire certificate chain, you must copy the Root CA + certificate and any intermediate certificates at the bottom of the chain. + A chain file has following structure: + + ```plaintext + -----BEGIN MY CERTIFICATE----- + -----END MY CERTIFICATE----- + -----BEGIN INTERMEDIATE CERTIFICATE----- + -----END INTERMEDIATE CERTIFICATE----- + -----BEGIN INTERMEDIATE CERTIFICATE----- + -----END INTERMEDIATE CERTIFICATE----- + -----BEGIN ROOT CERTIFICATE----- + -----END ROOT CERTIFICATE----- + ``` + + 1. **Token** - + GitLab authenticates against Kubernetes using service tokens, which are + scoped to a particular `namespace`. + **The token used should belong to a service account with + [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) + privileges.** To create this service account: + 1. Create a file called `gitlab-admin-service-account.yaml` with contents: + + ```yaml + apiVersion: v1 + kind: ServiceAccount + metadata: + name: gitlab + namespace: kube-system + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: gitlab-admin + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin + subjects: + - kind: ServiceAccount + name: gitlab + namespace: kube-system + ``` + + 1. Apply the service account and cluster role binding to your cluster: + + ```shell + kubectl apply -f gitlab-admin-service-account.yaml + ``` + + You need the `container.clusterRoleBindings.create` permission + to create cluster-level roles. If you do not have this permission, + you can alternatively enable Basic Authentication and then run the + `kubectl apply` command as an administrator: + + ```shell + kubectl apply -f gitlab-admin-service-account.yaml --username=admin --password=<password> + ``` + + NOTE: + Basic Authentication can be turned on and the password credentials + can be obtained using the Google Cloud Console. + + Output: + + ```shell + serviceaccount "gitlab" created + clusterrolebinding "gitlab-admin" created + ``` + + 1. Retrieve the token for the `gitlab` service account: + + ```shell + kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep gitlab | awk '{print $1}') + ``` + + Copy the `<authentication_token>` value from the output: + + ```plaintext + Name: gitlab-token-b5zv4 + Namespace: kube-system + Labels: <none> + Annotations: kubernetes.io/service-account.name=gitlab + kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8 + + Type: kubernetes.io/service-account-token + + Data + ==== + ca.crt: 1025 bytes + namespace: 11 bytes + token: <authentication_token> + ``` + + 1. **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. + See the [Managed clusters section](index.md#gitlab-managed-clusters) for more information. + 1. **Project namespace** (optional) - You don't have to fill this in. By leaving + it blank, GitLab creates one for you. Also: + - Each project should have a unique namespace. + - The project namespace is not necessarily the namespace of the secret, if + you're using a secret with broader permissions, like the secret from `default`. + - You should **not** use `default` as the project namespace. + - If you or someone created a secret specifically for the project, usually + with limited permissions, the secret's namespace and project namespace may + be the same. + +1. Select the **Add Kubernetes cluster** button. + +After about 10 minutes, your cluster is ready. + +## Disable Role-Based Access Control (RBAC) (optional) + +When connecting a cluster via GitLab integration, you may specify whether the +cluster is RBAC-enabled or not. This affects how GitLab interacts with the +cluster for certain operations. If you did *not* check the **RBAC-enabled cluster** +checkbox at creation time, GitLab assumes RBAC is disabled for your cluster +when interacting with it. If so, you must disable RBAC on your cluster for the +integration to work properly. + + + +WARNING: +Disabling RBAC means that any application running in the cluster, +or user who can authenticate to the cluster, has full API access. This is a +[security concern](index.md#security-implications), and may not be desirable. + +To effectively disable RBAC, global permissions can be applied granting full access: + +```shell +kubectl create clusterrolebinding permissive-binding \ + --clusterrole=cluster-admin \ + --user=admin \ + --user=kubelet \ + --group=system:serviceaccounts +``` diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index 9f0e5603785..a454b4dff99 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -4,7 +4,15 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Adding GKE clusters **(FREE)** +# GKE clusters (DEPRECATED) **(FREE)** + +> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/6049) in GitLab 14.0. + +WARNING: +Use [Infrastrucure as Code](../../infrastructure/index.md) to create new clusters. The method described in this document is deprecated as of GitLab 14.0. + +Through GitLab, you can create new clusters and add existing clusters hosted on Amazon Elastic +Kubernetes Service (EKS). GitLab supports adding new and existing GKE clusters. @@ -19,7 +27,12 @@ requirements are met: take up to 10 minutes after you create a project. For more information see the ["Before you begin" section of the Kubernetes Engine docs](https://cloud.google.com/kubernetes-engine/docs/quickstart#before-you-begin). -## New GKE cluster +## Add an existing GKE cluster + +If you already have a GKE cluster and want to integrate it with GitLab, +see how to [add an existing cluster](add_existing_cluster.md). + +## Create new GKE cluster Starting from [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/25925), all the GKE clusters provisioned by GitLab are [VPC-native](https://cloud.google.com/kubernetes-engine/docs/how-to/alias-ips). @@ -30,13 +43,13 @@ Note the following: at the instance level. If that's not the case, ask your GitLab administrator to enable it. On GitLab.com, this is enabled. - Starting from [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/55902), all GKE clusters - created by GitLab are RBAC-enabled. Take a look at the [RBAC section](add_remove_clusters.md#rbac-cluster-resources) for + created by GitLab are RBAC-enabled. Take a look at the [RBAC section](cluster_access.md#rbac-cluster-resources) for more information. - Starting from [GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18341), the cluster's pod address IP range is set to `/16` instead of the regular `/14`. `/16` is a CIDR notation. - GitLab requires basic authentication enabled and a client certificate issued for the cluster to - set up an [initial service account](add_remove_clusters.md#access-controls). In [GitLab versions + set up an [initial service account](cluster_access.md). In [GitLab versions 11.10 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58208), the cluster creation process explicitly requests GKE to create clusters with basic authentication enabled and a client certificate. @@ -49,14 +62,14 @@ To create and add a new Kubernetes cluster to your project, group, or instance: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - - **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. + - **Menu >** **{admin}** **Admin >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. 1. Click **Integrate with a cluster certificate**. 1. Under the **Create new cluster** tab, click **Google GKE**. 1. Connect your Google account if you haven't done already by clicking the **Sign in with Google** button. 1. Choose your cluster's settings: - **Kubernetes cluster name** - The name you wish to give the cluster. - - **Environment scope** - The [associated environment](index.md#setting-the-environment-scope) to this cluster. + - **Environment scope** - The [associated environment](multiple_kubernetes_clusters.md#setting-the-environment-scope) to this cluster. - **Google Cloud Platform project** - Choose the project you created in your GCP console to host the Kubernetes cluster. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). @@ -68,7 +81,7 @@ To create and add a new Kubernetes cluster to your project, group, or instance: - **Enable Cloud Run for Anthos** - Check this if you want to use Cloud Run for Anthos for this cluster. See the [Cloud Run for Anthos section](#cloud-run-for-anthos) for more information. - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. - See the [Managed clusters section](index.md#gitlab-managed-clusters) for more information. + See the [Managed clusters section](gitlab_managed_clusters.md) for more information. 1. Finally, click the **Create Kubernetes cluster** button. After a couple of minutes, your cluster is ready. @@ -81,8 +94,3 @@ You can choose to use Cloud Run for Anthos in place of installing Knative and Is separately after the cluster has been created. This means that Cloud Run (Knative), Istio, and HTTP Load Balancing are enabled on the cluster from the start, and cannot be installed or uninstalled. - -## Existing GKE cluster - -For information on adding an existing GKE cluster, see -[Existing Kubernetes cluster](add_remove_clusters.md#existing-kubernetes-cluster). diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index 2ecbc4a2ff5..6cada5648cb 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -4,28 +4,16 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Add a cluster using cluster certificates **(FREE)** +# Add a cluster using cluster certificates (DEPRECATED) **(FREE)** -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/6049) in GitLab 14.0. +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. WARNING: Creating a new cluster or adding an existing cluster to GitLab through the certificate-based method is deprecated and no longer recommended. Kubernetes cluster, similar to any other -infrastructure, should be created, updated, and maintained using [Infrastructure as Code](../../infrastructure/index.md). +infrastructure, should be created, updated, maintained using [Infrastructure as Code](../../infrastructure/index.md). GitLab is developing a built-in capability to create clusters with Terraform. -You can follow along in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/6049). - -GitLab offers integrated cluster creation for the following Kubernetes providers: - -- Google Kubernetes Engine (GKE). -- Amazon Elastic Kubernetes Service (EKS). - -GitLab can also integrate with any standard Kubernetes provider, either on-premise or hosted. - -NOTE: -Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) -and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) -in a few clicks. +You can follow along in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/6049). NOTE: Every new Google Cloud Platform (GCP) account receives @@ -35,351 +23,76 @@ accounts to get started with the GitLab integration with Google Kubernetes Engin [Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) to apply for credit. -## Before you begin - -Before [adding a Kubernetes cluster](#create-new-cluster) using GitLab, you need: - -- GitLab itself. Either: - - A [GitLab.com account](https://about.gitlab.com/pricing/#gitlab-com). - - A [self-managed installation](https://about.gitlab.com/pricing/#self-managed) with GitLab version - 12.5 or later. This ensures the GitLab UI can be used for cluster creation. -- The following GitLab access: - - [Maintainer role for a project](../../permissions.md#project-members-permissions) for a - project-level cluster. - - [Maintainer role for a group](../../permissions.md#group-members-permissions) for a - group-level cluster. - - [Admin Area access](../../admin_area/index.md) for a self-managed instance-level - cluster. **(FREE SELF)** - -## Access controls - -> - Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. - -When creating a cluster in GitLab, you are asked if you would like to create either: - -- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) - cluster, which is the GitLab default and recommended option. -- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster. - -When GitLab creates the cluster, -a `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace -to manage the newly created cluster. - -Helm also creates additional service accounts and other resources for each -installed application. Consult the documentation of the Helm charts for each application -for details. - -If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster), -ensure the token of the account has administrator privileges for the cluster. - -The resources created by GitLab differ depending on the type of cluster. - -### Important notes - -Note the following about access controls: - -- Environment-specific resources are only created if your cluster is - [managed by GitLab](index.md#gitlab-managed-clusters). -- If your cluster was created before GitLab 12.2, it uses a single namespace for all project - environments. - -### RBAC cluster resources - -GitLab creates the following resources for RBAC clusters. - -| Name | Type | Details | Created when | -|:----------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:-----------------------| -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | -| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | -| Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | -| Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | -| Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | -| Environment namespace | `RoleBinding` | [`admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | +NOTE: +Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) +and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) +in a few clicks. -The environment namespace `RoleBinding` was -[updated](https://gitlab.com/gitlab-org/gitlab/-/issues/31113) in GitLab 13.6 -to `admin` roleRef. Previously, the `edit` roleRef was used. +## Create new cluster -### ABAC cluster resources +> The certificate-based method for creating clusters from GitLab was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -GitLab creates the following resources for ABAC clusters. +As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/index.md) +to **safely create your new cluster from GitLab**. -| Name | Type | Details | Created when | -|:----------------------|:---------------------|:-------------------------------------|:---------------------------| -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | -| Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | -| Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | -| Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | +The certificate-based method is **deprecated** and scheduled for removal in +GitLab 15.0. However, you can still use it until then. Through +this method, you can host your cluster in EKS, GKE, on premises, and with other +providers. To host them on premises and with other providers, +use either the EKS or GKE method to guide you through and enter your cluster's +settings manually: -### Security of runners +- [New cluster hosted on Google Kubernetes Engine (GKE)](add_eks_clusters.md). +- [New cluster hosted on Amazon Elastic Kubernetes Service (EKS)](add_gke_clusters.md). -Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode) -enabled by default, which allows them to execute special commands and run -Docker in Docker. This functionality is needed to run some of the -[Auto DevOps](../../../topics/autodevops/index.md) -jobs. This implies the containers are running in privileged mode and you should, -therefore, be aware of some important details. +## Add existing cluster -The privileged flag gives all capabilities to the running container, which in -turn can do almost everything that the host can do. Be aware of the -inherent security risk associated with performing `docker run` operations on -arbitrary images as they effectively have root access. +If you already have a cluster and want to integrate it with GitLab, see how to +[add an existing cluster](add_existing_cluster.md). -If you don't want to use a runner in privileged mode, either: +## Configure your cluster -- Use shared runners on GitLab.com. They don't have this security issue. -- Set up your own runners using the configuration described at - [shared runners](../../gitlab_com/index.md#shared-runners) using - [`docker+machine`](https://docs.gitlab.com/runner/executors/docker_machine.html). +As of GitLab 14.0, use the [GitLab Kubernetes Agent](../../clusters/agent/index.md) to configure your cluster. -## Create new cluster +## Disable a cluster -New clusters can be created using GitLab on Google Kubernetes Engine (GKE) or -Amazon Elastic Kubernetes Service (EKS) at the project, group, or instance level: +When you successfully create a new Kubernetes cluster or add an existing +one to GitLab, the cluster connection to GitLab becomes enabled. To disable it: -1. Navigate to your: - - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level - cluster. +1. Go to your: + - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - - **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. -1. Click **Integrate with a cluster certificate**. -1. Click the **Create new cluster** tab. -1. Click either **Amazon EKS** or **Google GKE**, and follow the instructions for your desired service: - - [Amazon EKS](add_eks_clusters.md#new-eks-cluster). - - [Google GKE](add_gke_clusters.md#creating-the-cluster-on-gke). - -After creating a cluster, you can [install runners](https://docs.gitlab.com/runner/install/kubernetes.html), -add a [cluster management project](../../clusters/management_project.md), -configure [Auto DevOps](../../../topics/autodevops/index.md), -or start [deploying right away](index.md#deploying-to-a-kubernetes-cluster). - -## Add existing cluster - -If you have an existing Kubernetes cluster, you can add it to a project, group, -or instance, and [install runners](https://docs.gitlab.com/runner/install/kubernetes.html) -on it (the cluster does not need to be added to GitLab first). - -After adding a cluster, you can add a [cluster management project](../../clusters/management_project.md), -configure [Auto DevOps](../../../topics/autodevops/index.md), -or start [deploying right away](index.md#deploying-to-a-kubernetes-cluster). - -### Existing Kubernetes cluster - -To add a Kubernetes cluster to your project, group, or instance: - -1. Navigate to your: - 1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level - cluster. - 1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - 1. **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. -1. Click **Add Kubernetes cluster**. -1. Click the **Add existing cluster** tab and fill in the details: - 1. **Kubernetes cluster name** (required) - The name you wish to give the cluster. - 1. **Environment scope** (required) - The - [associated environment](index.md#setting-the-environment-scope) to this cluster. - 1. **API URL** (required) - - It's the URL that GitLab uses to access the Kubernetes API. Kubernetes - exposes several APIs, we want the "base" URL that is common to all of them. - For example, `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. - - Get the API URL by running this command: - - ```shell - kubectl cluster-info | grep -E 'Kubernetes master|Kubernetes control plane' | awk '/http/ {print $NF}' - ``` - - 1. **CA certificate** (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We use the certificate created by default. - 1. List the secrets with `kubectl get secrets`, and one should be named similar to - `default-token-xxxxx`. Copy that token name for use below. - 1. Get the certificate by running this command: - - ```shell - kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode - ``` - - If the command returns the entire certificate chain, you must copy the Root CA - certificate and any intermediate certificates at the bottom of the chain. - A chain file has following structure: - - ```plaintext - -----BEGIN MY CERTIFICATE----- - -----END MY CERTIFICATE----- - -----BEGIN INTERMEDIATE CERTIFICATE----- - -----END INTERMEDIATE CERTIFICATE----- - -----BEGIN INTERMEDIATE CERTIFICATE----- - -----END INTERMEDIATE CERTIFICATE----- - -----BEGIN ROOT CERTIFICATE----- - -----END ROOT CERTIFICATE----- - ``` - - 1. **Token** - - GitLab authenticates against Kubernetes using service tokens, which are - scoped to a particular `namespace`. - **The token used should belong to a service account with - [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) - privileges.** To create this service account: - 1. Create a file called `gitlab-admin-service-account.yaml` with contents: - - ```yaml - apiVersion: v1 - kind: ServiceAccount - metadata: - name: gitlab - namespace: kube-system - --- - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRoleBinding - metadata: - name: gitlab-admin - roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: cluster-admin - subjects: - - kind: ServiceAccount - name: gitlab - namespace: kube-system - ``` - - 1. Apply the service account and cluster role binding to your cluster: - - ```shell - kubectl apply -f gitlab-admin-service-account.yaml - ``` - - You need the `container.clusterRoleBindings.create` permission - to create cluster-level roles. If you do not have this permission, - you can alternatively enable Basic Authentication and then run the - `kubectl apply` command as an administrator: - - ```shell - kubectl apply -f gitlab-admin-service-account.yaml --username=admin --password=<password> - ``` - - NOTE: - Basic Authentication can be turned on and the password credentials - can be obtained using the Google Cloud Console. - - Output: - - ```shell - serviceaccount "gitlab" created - clusterrolebinding "gitlab-admin" created - ``` - - 1. Retrieve the token for the `gitlab` service account: - - ```shell - kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep gitlab | awk '{print $1}') - ``` - - Copy the `<authentication_token>` value from the output: - - ```plaintext - Name: gitlab-token-b5zv4 - Namespace: kube-system - Labels: <none> - Annotations: kubernetes.io/service-account.name=gitlab - kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8 - - Type: kubernetes.io/service-account-token - - Data - ==== - ca.crt: 1025 bytes - namespace: 11 bytes - token: <authentication_token> - ``` - - NOTE: - For GKE clusters, you need the - `container.clusterRoleBindings.create` permission to create a cluster - role binding. You can follow the [Google Cloud - documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access) - to grant access. - - 1. **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. - See the [Managed clusters section](index.md#gitlab-managed-clusters) for more information. - 1. **Project namespace** (optional) - You don't have to fill it in; by leaving - it blank, GitLab creates one for you. Also: - - Each project should have a unique namespace. - - The project namespace is not necessarily the namespace of the secret, if - you're using a secret with broader permissions, like the secret from `default`. - - You should **not** use `default` as the project namespace. - - If you or someone created a secret specifically for the project, usually - with limited permissions, the secret's namespace and project namespace may - be the same. - -1. Finally, click the **Create Kubernetes cluster** button. - -After a couple of minutes, your cluster is ready. - -#### Disable Role-Based Access Control (RBAC) (optional) - -When connecting a cluster via GitLab integration, you may specify whether the -cluster is RBAC-enabled or not. This affects how GitLab interacts with the -cluster for certain operations. If you did *not* check the **RBAC-enabled cluster** -checkbox at creation time, GitLab assumes RBAC is disabled for your cluster -when interacting with it. If so, you must disable RBAC on your cluster for the -integration to work properly. - - - -WARNING: -Disabling RBAC means that any application running in the cluster, -or user who can authenticate to the cluster, has full API access. This is a -[security concern](index.md#security-implications), and may not be desirable. - -To effectively disable RBAC, global permissions can be applied granting full access: - -```shell -kubectl create clusterrolebinding permissive-binding \ - --clusterrole=cluster-admin \ - --user=admin \ - --user=kubelet \ - --group=system:serviceaccounts -``` + - **Menu >** **{admin}** **Admin >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Select the name of the cluster you want to disable. +1. Toggle **GitLab Integration** off (in gray). +1. Click **Save changes**. -## Enabling or disabling integration +## Remove a cluster -The Kubernetes cluster integration enables after you have successfully either created -a new cluster or added an existing one. To disable Kubernetes cluster integration: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26815) in GitLab 12.6, you can remove cluster integrations and resources. -1. Navigate to your: - - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level - cluster. - - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - - **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. -1. Click on the name of the cluster. -1. Click the **GitLab Integration** toggle. -1. Click **Save changes**. +When you remove a cluster integration, you only remove the cluster relationship +to GitLab, not the cluster. To remove the cluster itself, visit your cluster's +GKE or EKS dashboard to do it from their UI or use `kubectl`. -## Removing integration +You need at least Maintainer [permissions](../../permissions.md) to your +project or group to remove the integration with GitLab. -To remove the Kubernetes cluster integration from your project, first navigate to the **Advanced Settings** tab of the cluster details page and either: +When removing a cluster integration, you have two options: -- Select **Remove integration**, to remove only the Kubernetes integration. -- [From GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/26815), select - **Remove integration and resources**, to also remove all related GitLab cluster resources (for - example, namespaces, roles, and bindings) when removing the integration. +- **Remove integration**: remove only the Kubernetes integration. +- **Remove integration and resources**: remove the cluster integration and +all GitLab cluster-related resources such as namespaces, roles, and bindings. -When removing the cluster integration, note: +To remove the Kubernetes cluster integration: -- You need Maintainer [permissions](../../permissions.md) and above to remove a Kubernetes cluster - integration. -- When you remove a cluster, you only remove its relationship to GitLab, not the cluster itself. To - remove the cluster, you can do so by visiting the GKE or EKS dashboard, or using `kubectl`. +1. Go to your cluster details page. +1. Select the **Advanced Settings** tab. +1. Select either **Remove integration** or **Remove integration and resources**. -## Learn more +## Access controls -To learn more on automatically deploying your applications, -read about [Auto DevOps](../../../topics/autodevops/index.md). +See [cluster access controls (RBAC or ABAC)](cluster_access.md). ## Troubleshooting diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md new file mode 100644 index 00000000000..713a60b2dd0 --- /dev/null +++ b/doc/user/project/clusters/cluster_access.md @@ -0,0 +1,88 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Cluster access controls (RBAC or ABAC) + +> Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. + +When creating a cluster in GitLab, you are asked if you would like to create either: + +- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) + cluster, which is the GitLab default and recommended option. +- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster. + +When GitLab creates the cluster, +a `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace +to manage the newly created cluster. + +Helm also creates additional service accounts and other resources for each +installed application. Consult the documentation of the Helm charts for each application +for details. + +If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster), +ensure the token of the account has administrator privileges for the cluster. + +The resources created by GitLab differ depending on the type of cluster. + +## Important notes + +Note the following about access controls: + +- Environment-specific resources are only created if your cluster is + [managed by GitLab](index.md#gitlab-managed-clusters). +- If your cluster was created before GitLab 12.2, it uses a single namespace for all project + environments. + +## RBAC cluster resources + +GitLab creates the following resources for RBAC clusters. + +| Name | Type | Details | Created when | +|:----------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:-----------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | +| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | +| Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | +| Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | +| Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | +| Environment namespace | `RoleBinding` | [`admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | + +The environment namespace `RoleBinding` was +[updated](https://gitlab.com/gitlab-org/gitlab/-/issues/31113) in GitLab 13.6 +to `admin` roleRef. Previously, the `edit` roleRef was used. + +## ABAC cluster resources + +GitLab creates the following resources for ABAC clusters. + +| Name | Type | Details | Created when | +|:----------------------|:---------------------|:-------------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | +| Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | +| Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | +| Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | + +## Security of runners + +Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode) +enabled by default, which allows them to execute special commands and run +Docker in Docker. This functionality is needed to run some of the +[Auto DevOps](../../../topics/autodevops/index.md) +jobs. This implies the containers are running in privileged mode and you should, +therefore, be aware of some important details. + +The privileged flag gives all capabilities to the running container, which in +turn can do almost everything that the host can do. Be aware of the +inherent security risk associated with performing `docker run` operations on +arbitrary images as they effectively have root access. + +If you don't want to use a runner in privileged mode, either: + +- Use shared runners on GitLab.com. They don't have this security issue. +- Set up your own runners using the configuration described at +[shared runners](../../gitlab_com/index.md#shared-runners) using +[`docker+machine`](https://docs.gitlab.com/runner/executors/docker_machine.html). diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md new file mode 100644 index 00000000000..fdd65d70242 --- /dev/null +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -0,0 +1,141 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Deploy to a Kubernetes cluster + +A Kubernetes cluster can be the destination for a deployment job. If + +- The cluster is integrated with GitLab, special + [deployment variables](#deployment-variables) are made available to your job + and configuration is not required. You can immediately begin interacting with + the cluster from your jobs using tools such as `kubectl` or `helm`. +- You don't use the GitLab cluster integration, you can still deploy to your + cluster. However, you must configure Kubernetes tools yourself + using [CI/CD variables](../../../ci/variables/index.md#custom-cicd-variables) + before you can interact with the cluster from your jobs. + +## Deployment variables + +Deployment variables require a valid [Deploy Token](../deploy_tokens/index.md) named +[`gitlab-deploy-token`](../deploy_tokens/index.md#gitlab-deploy-token), and the +following command in your deployment job script, for Kubernetes to access the registry: + +- Using Kubernetes 1.18+: + + ```shell + kubectl create secret docker-registry gitlab-registry --docker-server="$CI_REGISTRY" --docker-username="$CI_DEPLOY_USER" --docker-password="$CI_DEPLOY_PASSWORD" --docker-email="$GITLAB_USER_EMAIL" -o yaml --dry-run=client | kubectl apply -f - + ``` + +- Using Kubernetes <1.18: + + ```shell + kubectl create secret docker-registry gitlab-registry --docker-server="$CI_REGISTRY" --docker-username="$CI_DEPLOY_USER" --docker-password="$CI_DEPLOY_PASSWORD" --docker-email="$GITLAB_USER_EMAIL" -o yaml --dry-run | kubectl apply -f - + ``` + +The Kubernetes cluster integration exposes these +[deployment variables](../../../ci/variables/index.md#deployment-variables) in the +GitLab CI/CD build environment to deployment jobs. Deployment jobs have +[defined a target environment](../../../ci/environments/index.md). + +| Deployment Variable | Description | +|----------------------------|-------------| +| `KUBE_URL` | Equal to the API URL. | +| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](cluster_access.md). Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main service account of the cluster integration. | +| `KUBE_NAMESPACE` | The namespace associated with the project's deployment service account. In the format `<project_name>-<project_id>-<environment>`. For GitLab-managed clusters, a matching namespace is automatically created by GitLab in the cluster. If your cluster was created before GitLab 12.2, the default `KUBE_NAMESPACE` is set to `<project_name>-<project_id>`. | +| `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. | +| `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. | +| `KUBECONFIG` | Path to a file containing `kubeconfig` for this deployment. CA bundle would be embedded if specified. This configuration also embeds the same token defined in `KUBE_TOKEN` so you likely need only this variable. This variable name is also automatically picked up by `kubectl` so you don't need to reference it explicitly if using `kubectl`. | +| `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, this variable can be used to set a domain per cluster. See [cluster domains](gitlab_managed_clusters.md#base-domain) for more information. | + +## Custom namespace + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. +> - An option to use project-wide namespaces [was added](https://gitlab.com/gitlab-org/gitlab/-/issues/38054) in GitLab 13.5. + +The Kubernetes integration provides a `KUBECONFIG` with an auto-generated namespace +to deployment jobs. It defaults to using project-environment specific namespaces +of the form `<prefix>-<environment>`, where `<prefix>` is of the form +`<project_name>-<project_id>`. To learn more, read [Deployment variables](#deployment-variables). + +You can customize the deployment namespace in a few ways: + +- You can choose between a **namespace per [environment](../../../ci/environments/index.md)** + or a **namespace per project**. A namespace per environment is the default and recommended + setting, as it prevents the mixing of resources between production and non-production environments. +- When using a project-level cluster, you can additionally customize the namespace prefix. + When using namespace-per-environment, the deployment namespace is `<prefix>-<environment>`, + but otherwise just `<prefix>`. +- For **non-managed** clusters, the auto-generated namespace is set in the `KUBECONFIG`, + but the user is responsible for ensuring its existence. You can fully customize + this value using + [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configure-kubernetes-deployments) + in `.gitlab-ci.yml`. + +When you customize the namespace, existing environments remain linked to their current +namespaces until you [clear the cluster cache](gitlab_managed_clusters.md#clearing-the-cluster-cache). + +### Protecting credentials + +By default, anyone who can create a deployment job can access any CI/CD variable in +an environment's deployment job. This includes `KUBECONFIG`, which gives access to +any secret available to the associated service account in your cluster. +To keep your production credentials safe, consider using +[protected environments](../../../ci/environments/protected_environments.md), +combined with *one* of the following: + +- A GitLab-managed cluster and namespace per environment. +- An environment-scoped cluster per protected environment. The same cluster + can be added multiple times with multiple restricted service accounts. + +## Web terminals for Kubernetes clusters + +> Introduced in GitLab 8.15. + +The Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) +support to your [environments](../../../ci/environments/index.md). This is based +on the `exec` functionality found in Docker and Kubernetes, so you get a new +shell session in your existing containers. To use this integration, you +should deploy to Kubernetes using the deployment variables above, ensuring any +deployments, replica sets, and pods are annotated with: + +- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` +- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` + +`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of +the CI/CD variables. + +You must be the project owner or have `maintainer` permissions to use terminals. +Support is limited to the first container in the first pod of your environment. + +## Troubleshooting + +Before the deployment jobs starts, GitLab creates the following specifically for +the deployment job: + +- A namespace. +- A service account. + +However, sometimes GitLab cannot create them. In such instances, your job can fail with the message: + +```plaintext +This job failed because the necessary resources were not successfully created. +``` + +To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs.md#kuberneteslog). + +Reasons for failure include: + +- The token you gave GitLab does not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) + privileges required by GitLab. +- Missing `KUBECONFIG` or `KUBE_TOKEN` deployment variables. To be passed to your job, they must have a matching + [`environment:name`](../../../ci/environments/index.md). If your job has no + `environment:name` set, the Kubernetes credentials are not passed to it. + +NOTE: +Project-level clusters upgraded from GitLab 12.0 or older may be configured +in a way that causes this error. Ensure you deselect the +[GitLab-managed cluster](gitlab_managed_clusters.md) option if you want to manage +namespaces and service accounts yourself. diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md new file mode 100644 index 00000000000..77921ec1dff --- /dev/null +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -0,0 +1,102 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# GitLab-managed clusters + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. +> - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. + +You can choose to allow GitLab to manage your cluster for you. If your cluster +is managed by GitLab, resources for your projects are automatically created. See +the [Access controls](add_remove_clusters.md#access-controls) section for +details about the created resources. + +If you choose to manage your own cluster, project-specific resources aren't created +automatically. If you are using [Auto DevOps](../../../topics/autodevops/index.md), you must +explicitly provide the `KUBE_NAMESPACE` [deployment variable](deploy_to_cluster.md#deployment-variables) +for your deployment jobs to use. Otherwise, a namespace is created for you. + +WARNING: +Be aware that manually managing resources that have been created by GitLab, like +namespaces and service accounts, can cause unexpected errors. If this occurs, try +[clearing the cluster cache](#clearing-the-cluster-cache). + +## Clearing the cluster cache + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31759) in GitLab 12.6. + +If you allow GitLab to manage your cluster, GitLab stores a cached +version of the namespaces and service accounts it creates for your projects. If you +modify these resources in your cluster manually, this cache can fall out of sync with +your cluster. This can cause deployment jobs to fail. + +To clear the cache: + +1. Navigate to your project's **Infrastructure > Kubernetes clusters** page, and select your cluster. +1. Expand the **Advanced settings** section. +1. Click **Clear cluster cache**. + +## Base domain + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8. + +Specifying a base domain automatically sets `KUBE_INGRESS_BASE_DOMAIN` as an deployment variable. +If you are using [Auto DevOps](../../../topics/autodevops/index.md), this domain is used for the different +stages. For example, Auto Review Apps and Auto Deploy. + +The domain should have a wildcard DNS configured to the Ingress IP address. +You can either: + +- Create an `A` record that points to the Ingress IP address with your domain provider. +- Enter a wildcard DNS address using a service such as `nip.io` or `xip.io`. For example, `192.168.1.1.xip.io`. + +To determine the external Ingress IP address, or external Ingress hostname: + +- *If the cluster is on GKE*: + 1. Click the **Google Kubernetes Engine** link in the **Advanced settings**, + or go directly to the [Google Kubernetes Engine dashboard](https://console.cloud.google.com/kubernetes/). + 1. Select the proper project and cluster. + 1. Click **Connect** + 1. Execute the `gcloud` command in a local terminal or using the **Cloud Shell**. + +- *If the cluster is not on GKE*: Follow the specific instructions for your + Kubernetes provider to configure `kubectl` with the right credentials. + The output of the following examples show the external endpoint of your + cluster. This information can then be used to set up DNS entries and forwarding + rules that allow external access to your deployed applications. + +Depending an your Ingress, the external IP address can be retrieved in various ways. +This list provides a generic solution, and some GitLab-specific approaches: + +- In general, you can list the IP addresses of all load balancers by running: + + ```shell + kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' + ``` + +- If you installed Ingress using the **Applications**, run: + + ```shell + kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' + ``` + +- Some Kubernetes clusters return a hostname instead, like + [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: + + ```shell + kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' + ``` + + If you use EKS, an [Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/) + is also created, which incurs additional AWS costs. + +- Istio/Knative uses a different command. Run: + + ```shell + kubectl get svc --namespace=istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' + ``` + +If you see a trailing `%` on some Kubernetes versions, do not include it. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 8dd8ed52dd7..a0efea267f0 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,6 +1,6 @@ --- -stage: Monitor -group: Monitor +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -12,35 +12,30 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in > GitLab 11.11 for [instances](../../instance/clusters/index.md). -Using the GitLab project Kubernetes integration, you can: - -- Use [Review Apps](../../../ci/review_apps/index.md). -- Run [pipelines](../../../ci/pipelines/index.md). -- [Deploy](#deploying-to-a-kubernetes-cluster) your applications. -- Detect and [monitor Kubernetes](#monitoring-your-kubernetes-cluster). -- Use it with [Auto DevOps](#auto-devops). -- Use [Web terminals](#web-terminals). -- Use [Deploy Boards](#deploy-boards). -- Use [Canary Deployments](#canary-deployments). **(PREMIUM)** -- Use [deployment variables](#deployment-variables). -- Use [role-based or attribute-based access controls](add_remove_clusters.md#access-controls). -- View [Logs](#viewing-pod-logs). -- Run serverless workloads on [Kubernetes with Knative](serverless/index.md). +We offer extensive integrations to help you connect and manage your Kubernetes clusters from GitLab. -Besides integration at the project level, Kubernetes clusters can also be -integrated at the [group level](../../group/clusters/index.md) or -[GitLab instance level](../../instance/clusters/index.md). +Read through this document to get started. -To view your project level Kubernetes clusters, navigate to **Infrastructure > Kubernetes clusters** -from your project. On this page, you can [add a new cluster](#adding-and-removing-clusters) -and view information about your existing clusters, such as: +## Clusters infrastructure -- Nodes count. -- Rough estimates of memory and CPU usage. +Use [Infrastructure as Code](../../infrastructure) to create and manage your clusters with the GitLab integration with Terraform. + +## Benefit from the GitLab-Kubernetes integration + +Using the GitLab-Kubernetes integration, you can benefit of GitLab +features such as: -## Setting up +- Create [CI/CD Pipelines](../../../ci/pipelines/index.md) to build, test, and deploy to your cluster. +- Use [Auto DevOps](#auto-devops) to automate the CI/CD process. +- Use [role-based or attribute-based access controls](cluster_access.md). +- Run serverless workloads on [Kubernetes with Knative](serverless/index.md). +- Connect GitLab to in-cluster applications using [cluster integrations](../../clusters/integrations.md). +- Use [Deploy Boards](../deploy_boards.md) to see the health and status of each CI [environment](../../../ci/environments/index.md) running on your Kubernetes cluster. +- Use [Canary deployments](../canary_deployments.md) to update only a portion of your fleet with the latest version of your application. +- View your [Kubernetes podlogs](kubernetes_pod_logs.md) directly in GitLab. +- Connect to your cluster through GitLab [web terminals](deploy_to_cluster.md#web-terminals-for-kubernetes-clusters). -### Supported cluster versions +## Supported cluster versions GitLab is committed to support at least two production-ready Kubernetes minor versions at any given time. We regularly review the versions we support, and @@ -48,7 +43,7 @@ provide a three-month deprecation period before we remove support of a specific version. The range of supported versions is based on the evaluation of: - The versions supported by major managed Kubernetes providers. -- The versions [supported by the Kubernetes community](https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions). +- The versions [supported by the Kubernetes community](https://kubernetes.io/releases/version-skew-policy/#supported-versions). GitLab supports the following Kubernetes versions, and you can upgrade your Kubernetes version to any supported version at any time: @@ -61,88 +56,34 @@ Kubernetes version to any supported version at any time: Some GitLab features may support versions outside the range provided here. -NOTE: -[GKE Cluster creation](add_remove_clusters.md#create-new-cluster) by GitLab is currently not supported for Kubernetes 1.19+. For these versions you can create the cluster through GCP, then [Add existing cluster](add_remove_clusters.md#add-existing-cluster). See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331922) for more information. - -### Adding and removing clusters - -See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how -to: - -- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service - (EKS) using the GitLab UI. -- Add an integration to an existing cluster from any Kubernetes platform. - -### Multiple Kubernetes clusters - -> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3 -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab Free in 13.2. +## Add and remove clusters -You can associate more than one Kubernetes cluster to your -project. That way you can have different clusters for different environments, -like development, staging, production, and so on. -Add another cluster, like you did the first time, and make sure to -[set an environment scope](#setting-the-environment-scope) that -differentiates the new cluster from the rest. +You can create new or add existing clusters to GitLab: -#### Setting the environment scope +- On the project-level, to have a cluster dedicated to a project. +- On the [group level](../../group/clusters/index.md), to use the same cluster across multiple projects within your group. +- On the [instance level](../../instance/clusters/index.md), to use the same cluster across multiple groups and projects. **(FREE SELF)** -When adding more than one Kubernetes cluster to your project, you need to differentiate -them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the -[environment-specific CI/CD variables](../../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable) work. +To create new clusters, use one of the following methods: -The default environment scope is `*`, which means all jobs, regardless of their -environment, use that cluster. Each scope can be used only by a single cluster -in a project, and a validation error occurs if otherwise. Also, jobs that don't -have an environment keyword set can't access any cluster. +- [Infrastructure as Code](../../infrastructure/index.md) (**recommended**). +- [Cluster certificates](add_remove_clusters.md) (**deprecated**). -For example, let's say the following Kubernetes clusters exist in a project: +You can also [add existing clusters](add_existing_cluster.md) to GitLab. -| Cluster | Environment scope | -| ----------- | ----------------- | -| Development | `*` | -| Production | `production` | +## View your clusters -And the following environments are set in -[`.gitlab-ci.yml`](../../../ci/yaml/README.md): - -```yaml -stages: - - test - - deploy - -test: - stage: test - script: sh test - -deploy to staging: - stage: deploy - script: make deploy - environment: - name: staging - url: https://staging.example.com/ - -deploy to production: - stage: deploy - script: make deploy - environment: - name: production - url: https://example.com/ -``` - -The results: +To view your project-level Kubernetes clusters, to go **Infrastructure > Kubernetes clusters** +from your project. On this page, you can add a new cluster +and view information about your existing clusters, such as: -- The Development cluster details are available in the `deploy to staging` - job. -- The production cluster details are available in the `deploy to production` - job. -- No cluster details are available in the `test` job because it doesn't - define any environment. +- Nodes count. +- Rough estimates of memory and CPU usage. ## Configuring your Kubernetes cluster -After [adding a Kubernetes cluster](add_remove_clusters.md) to GitLab, read this section that covers -important considerations for configuring Kubernetes clusters with GitLab. +Use the [GitLab Kubernetes Agent](../../clusters/agent/index.md) to safely +configure your clusters. Otherwise, there are [security implications](#security-implications). ### Security implications @@ -155,103 +96,15 @@ functionalities needed to successfully build and deploy a containerized application. Bear in mind that the same credentials are used for all the applications running on the cluster. -### GitLab-managed clusters - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. -> - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. - -You can choose to allow GitLab to manage your cluster for you. If your cluster -is managed by GitLab, resources for your projects are automatically created. See -the [Access controls](add_remove_clusters.md#access-controls) section for -details about the created resources. - -If you choose to manage your own cluster, project-specific resources aren't created -automatically. If you are using [Auto DevOps](../../../topics/autodevops/index.md), you must -explicitly provide the `KUBE_NAMESPACE` [deployment variable](#deployment-variables) -for your deployment jobs to use. Otherwise, a namespace is created for you. - -#### Important notes - -Be aware that manually managing resources that have been created by GitLab, like -namespaces and service accounts, can cause unexpected errors. If this occurs, try -[clearing the cluster cache](#clearing-the-cluster-cache). - -#### Clearing the cluster cache - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31759) in GitLab 12.6. - -If you allow GitLab to manage your cluster, GitLab stores a cached -version of the namespaces and service accounts it creates for your projects. If you -modify these resources in your cluster manually, this cache can fall out of sync with -your cluster. This can cause deployment jobs to fail. - -To clear the cache: - -1. Navigate to your project's **Infrastructure > Kubernetes clusters** page, and select your cluster. -1. Expand the **Advanced settings** section. -1. Click **Clear cluster cache**. - -### Base domain - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8. - -Specifying a base domain automatically sets `KUBE_INGRESS_BASE_DOMAIN` as an deployment variable. -If you are using [Auto DevOps](../../../topics/autodevops/index.md), this domain is used for the different -stages. For example, Auto Review Apps and Auto Deploy. - -The domain should have a wildcard DNS configured to the Ingress IP address. -You can either: - -- Create an `A` record that points to the Ingress IP address with your domain provider. -- Enter a wildcard DNS address using a service such as `nip.io` or `xip.io`. For example, `192.168.1.1.xip.io`. - -To determine the external Ingress IP address, or external Ingress hostname: - -- *If the cluster is on GKE*: - 1. Click the **Google Kubernetes Engine** link in the **Advanced settings**, - or go directly to the [Google Kubernetes Engine dashboard](https://console.cloud.google.com/kubernetes/). - 1. Select the proper project and cluster. - 1. Click **Connect** - 1. Execute the `gcloud` command in a local terminal or using the **Cloud Shell**. - -- *If the cluster is not on GKE*: Follow the specific instructions for your - Kubernetes provider to configure `kubectl` with the right credentials. - The output of the following examples show the external endpoint of your - cluster. This information can then be used to set up DNS entries and forwarding - rules that allow external access to your deployed applications. - -Depending an your Ingress, the external IP address can be retrieved in various ways. -This list provides a generic solution, and some GitLab-specific approaches: - -- In general, you can list the IP addresses of all load balancers by running: - - ```shell - kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' - ``` - -- If you installed Ingress using the **Applications**, run: - - ```shell - kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' - ``` - -- Some Kubernetes clusters return a hostname instead, like - [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: - - ```shell - kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' - ``` - - If you use EKS, an [Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/) - is also created, which incurs additional AWS costs. +## Multiple Kubernetes clusters -- Istio/Knative uses a different command. Run: +See how to associate [multiple Kubernetes clusters](multiple_kubernetes_clusters.md) +with your GitLab project. - ```shell - kubectl get svc --namespace=istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' - ``` +## Cluster integrations -If you see a trailing `%` on some Kubernetes versions, do not include it. +See the available [cluster integrations](../../clusters/integrations.md) +to integrate third-party applications with your clusters through GitLab. ## Cluster management project @@ -259,180 +112,18 @@ Attach a [Cluster management project](../../clusters/management_project.md) to your cluster to manage shared resources requiring `cluster-admin` privileges for installation, such as an Ingress controller. -## Auto DevOps +## GitLab-managed clusters -Auto DevOps automatically detects, builds, tests, deploys, and monitors your -applications. +See how to allow [GitLab to manage your cluster for you](gitlab_managed_clusters.md). -To make full use of Auto DevOps (Auto Deploy, Auto Review Apps, and -Auto Monitoring) the Kubernetes project integration must be enabled. However, -Kubernetes clusters can be used without Auto DevOps. +## Auto DevOps -[Read more about Auto DevOps](../../../topics/autodevops/index.md). +You can use [Auto DevOps](../../../topics/autodevops/index.md) to automatically +detect, build, test, deploy, and monitor your applications. ## Deploying to a Kubernetes cluster -A Kubernetes cluster can be the destination for a deployment job. If - -- The cluster is integrated with GitLab, special - [deployment variables](#deployment-variables) are made available to your job - and configuration is not required. You can immediately begin interacting with - the cluster from your jobs using tools such as `kubectl` or `helm`. -- You don't use the GitLab cluster integration, you can still deploy to your - cluster. However, you must configure Kubernetes tools yourself - using [CI/CD variables](../../../ci/variables/README.md#custom-cicd-variables) - before you can interact with the cluster from your jobs. - -### Deployment variables - -Deployment variables require a valid [Deploy Token](../deploy_tokens/index.md) named -[`gitlab-deploy-token`](../deploy_tokens/index.md#gitlab-deploy-token), and the -following command in your deployment job script, for Kubernetes to access the registry: - -- Using Kubernetes 1.18+: - - ```shell - kubectl create secret docker-registry gitlab-registry --docker-server="$CI_REGISTRY" --docker-username="$CI_DEPLOY_USER" --docker-password="$CI_DEPLOY_PASSWORD" --docker-email="$GITLAB_USER_EMAIL" -o yaml --dry-run=client | kubectl apply -f - - ``` - -- Using Kubernetes <1.18: - - ```shell - kubectl create secret docker-registry gitlab-registry --docker-server="$CI_REGISTRY" --docker-username="$CI_DEPLOY_USER" --docker-password="$CI_DEPLOY_PASSWORD" --docker-email="$GITLAB_USER_EMAIL" -o yaml --dry-run | kubectl apply -f - - ``` - -The Kubernetes cluster integration exposes these -[deployment variables](../../../ci/variables/README.md#deployment-variables) in the -GitLab CI/CD build environment to deployment jobs. Deployment jobs have -[defined a target environment](../../../ci/environments/index.md). - -| Deployment Variable | Description | -|----------------------------|-------------| -| `KUBE_URL` | Equal to the API URL. | -| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](add_remove_clusters.md#access-controls). Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main service account of the cluster integration. | -| `KUBE_NAMESPACE` | The namespace associated with the project's deployment service account. In the format `<project_name>-<project_id>-<environment>`. For GitLab-managed clusters, a matching namespace is automatically created by GitLab in the cluster. If your cluster was created before GitLab 12.2, the default `KUBE_NAMESPACE` is set to `<project_name>-<project_id>`. | -| `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. | -| `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. | -| `KUBECONFIG` | Path to a file containing `kubeconfig` for this deployment. CA bundle would be embedded if specified. This configuration also embeds the same token defined in `KUBE_TOKEN` so you likely need only this variable. This variable name is also automatically picked up by `kubectl` so you don't need to reference it explicitly if using `kubectl`. | -| `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, this variable can be used to set a domain per cluster. See [cluster domains](#base-domain) for more information. | - -### Custom namespace - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. -> - An option to use project-wide namespaces [was added](https://gitlab.com/gitlab-org/gitlab/-/issues/38054) in GitLab 13.5. - -The Kubernetes integration provides a `KUBECONFIG` with an auto-generated namespace -to deployment jobs. It defaults to using project-environment specific namespaces -of the form `<prefix>-<environment>`, where `<prefix>` is of the form -`<project_name>-<project_id>`. To learn more, read [Deployment variables](#deployment-variables). - -You can customize the deployment namespace in a few ways: - -- You can choose between a **namespace per [environment](../../../ci/environments/index.md)** - or a **namespace per project**. A namespace per environment is the default and recommended - setting, as it prevents the mixing of resources between production and non-production environments. -- When using a project-level cluster, you can additionally customize the namespace prefix. - When using namespace-per-environment, the deployment namespace is `<prefix>-<environment>`, - but otherwise just `<prefix>`. -- For **non-managed** clusters, the auto-generated namespace is set in the `KUBECONFIG`, - but the user is responsible for ensuring its existence. You can fully customize - this value using - [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configure-kubernetes-deployments) - in `.gitlab-ci.yml`. - -When you customize the namespace, existing environments remain linked to their current -namespaces until you [clear the cluster cache](#clearing-the-cluster-cache). - -#### Protecting credentials - -By default, anyone who can create a deployment job can access any CI/CD variable in -an environment's deployment job. This includes `KUBECONFIG`, which gives access to -any secret available to the associated service account in your cluster. -To keep your production credentials safe, consider using -[protected environments](../../../ci/environments/protected_environments.md), -combined with *one* of the following: - -- A GitLab-managed cluster and namespace per environment. -- An environment-scoped cluster per protected environment. The same cluster - can be added multiple times with multiple restricted service accounts. - -### Integrations - -#### Canary Deployments - -Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) -and visualize your canary deployments right inside the Deploy Board, without -the need to leave GitLab. - -[Read more about Canary Deployments](../canary_deployments.md) - -#### Deploy Boards - -GitLab Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes. -They display the status of the pods in the deployment. Developers and other -teammates can view the progress and status of a rollout, pod by pod, in the -workflow they already use without any need to access Kubernetes. - -[Read more about Deploy Boards](../deploy_boards.md) - -#### Viewing pod logs - -GitLab enables you to view the logs of running pods in connected Kubernetes -clusters. By displaying the logs directly in GitLab, developers can avoid having -to manage console tools or jump to a different interface. - -[Read more about Kubernetes logs](kubernetes_pod_logs.md) - -#### Web terminals - -> Introduced in GitLab 8.15. - -When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) -support to your [environments](../../../ci/environments/index.md). This is based -on the `exec` functionality found in Docker and Kubernetes, so you get a new -shell session in your existing containers. To use this integration, you -should deploy to Kubernetes using the deployment variables above, ensuring any -deployments, replica sets, and pods are annotated with: - -- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` -- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` - -`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of -the CI/CD variables. - -You must be the project owner or have `maintainer` permissions to use terminals. -Support is limited to the first container in the first pod of your environment. - -### Troubleshooting - -Before the deployment jobs starts, GitLab creates the following specifically for -the deployment job: - -- A namespace. -- A service account. - -However, sometimes GitLab can not create them. In such instances, your job can fail with the message: - -```plaintext -This job failed because the necessary resources were not successfully created. -``` - -To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs.md#kuberneteslog). - -Reasons for failure include: - -- The token you gave GitLab does not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) - privileges required by GitLab. -- Missing `KUBECONFIG` or `KUBE_TOKEN` deployment variables. To be passed to your job, they must have a matching - [`environment:name`](../../../ci/environments/index.md). If your job has no - `environment:name` set, the Kubernetes credentials are not passed to it. - -NOTE: -Project-level clusters upgraded from GitLab 12.0 or older may be configured -in a way that causes this error. Ensure you deselect the -[GitLab-managed cluster](#gitlab-managed-clusters) option if you want to manage -namespaces and service accounts yourself. +See how to [deploy to your Kubernetes cluster](deploy_to_cluster.md) from GitLab. ## Monitoring your Kubernetes cluster diff --git a/doc/user/project/clusters/multiple_kubernetes_clusters.md b/doc/user/project/clusters/multiple_kubernetes_clusters.md new file mode 100644 index 00000000000..e2eae011b8c --- /dev/null +++ b/doc/user/project/clusters/multiple_kubernetes_clusters.md @@ -0,0 +1,71 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Multiple Kubernetes clusters for a single project + +> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3 +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab Free in 13.2. + +You can associate more than one Kubernetes cluster to your +project. That way you can have different clusters for different environments, +like development, staging, production, and so on. +Add another cluster, like you did the first time, and make sure to +[set an environment scope](#setting-the-environment-scope) that +differentiates the new cluster from the rest. + +## Setting the environment scope + +When adding more than one Kubernetes cluster to your project, you need to differentiate +them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the +[environment-specific CI/CD variables](../../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable) work. + +The default environment scope is `*`, which means all jobs, regardless of their +environment, use that cluster. Each scope can be used only by a single cluster +in a project, and a validation error occurs if otherwise. Also, jobs that don't +have an environment keyword set can't access any cluster. + +For example, let's say the following Kubernetes clusters exist in a project: + +| Cluster | Environment scope | +| ----------- | ----------------- | +| Development | `*` | +| Production | `production` | + +And the following environments are set in +[`.gitlab-ci.yml`](../../../ci/yaml/index.md): + +```yaml +stages: + - test + - deploy + +test: + stage: test + script: sh test + +deploy to staging: + stage: deploy + script: make deploy + environment: + name: staging + url: https://staging.example.com/ + +deploy to production: + stage: deploy + script: make deploy + environment: + name: production + url: https://example.com/ +``` + +The results: + +- The Development cluster details are available in the `deploy to staging` + job. +- The production cluster details are available in the `deploy to production` + job. +- No cluster details are available in the `test` job because it doesn't + define any environment. diff --git a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md index ebcd56078ae..466bcb7916f 100644 --- a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md +++ b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md @@ -20,7 +20,7 @@ The following steps are recommended to install and use Container Host Security t 1. Install and configure an Ingress node: - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../index.md#base-domain) + - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes cluster. @@ -50,7 +50,7 @@ kubectl -n gitlab-managed-apps logs -l app=falco Your CI/CD pipeline may occasionally fail or have trouble connecting to the cluster. Here are some initial troubleshooting steps that resolve the most common problems: -1. [Clear the cluster cache](../../index.md#clearing-the-cluster-cache) +1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache) 1. If things still aren't working, a more assertive set of actions may help get things back to a good state: diff --git a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md index 33aefec224a..62010bb7802 100644 --- a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md +++ b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md @@ -20,20 +20,114 @@ The following steps are recommended to install and use Container Network Securit 1. Install and configure an Ingress node: - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../index.md#base-domain) + - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes cluster. -1. [Install and configure Cilium](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd). +1. [Install and configure Cilium](#use-the-cluster-management-template-to-install-cilium). 1. Be sure to restart all pods that were running before Cilium was installed by running this command in your cluster: `kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod` + You can skip this step if `nodeinit.restartPods` is set to `true` on your Helm chart. + It's possible to install and manage Cilium in other ways. For example, you could use the GitLab Helm chart to install Cilium manually in a Kubernetes cluster, and then connect it back to GitLab. However, such methods aren't documented or officially supported by GitLab. +### Use the Cluster Management template to install Cilium + +[Cilium](https://cilium.io/) is a networking plug-in for Kubernetes that you can use to implement +support for [`NetworkPolicy`](https://kubernetes.io/docs/concepts/services-networking/network-policies/) +resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). + +You can use the [Cluster Management Project Template](../../../../clusters/management_project_template.md) +to install Cilium in your Kubernetes cluster. + +1. In your cluster management project, go to `helmfile.yaml` and uncomment `- path: applications/cilium/helmfile.yaml`. +1. In `applications/cilium/helmfile.yaml`, set `clusterType` to either `gke` or `eks` based on which Kubernetes provider your are using. + + ```yaml + environments: + default: + values: + # Set to "gke" or "eks" based on your cluster type + - clusterType: "" + ``` + +1. Merge or push these changes to the default branch of your cluster management project, +and [GitLab CI/CD](../../../../../ci/README.md) will automatically install Cilium. + +WARNING: +Installation and removal of the Cilium requires a **manual** +[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) +of all affected pods in all namespaces to ensure that they are +[managed](https://docs.cilium.io/en/stable/operations/troubleshooting/#ensure-managed-pod) +by the correct networking plug-in. When Hubble is enabled, its related pod might require a +restart depending on whether it started prior to Cilium. For more information, see +[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) +in the Kubernetes docs. + +NOTE: +Major upgrades might require additional setup steps. For more information, see +the official [upgrade guide](https://docs.cilium.io/en/stable/operations/upgrade/). + +Support for installing the Cilium application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). + +### Configure the Cilium Helm chart + +You can customize Cilium's Helm variables by editing the `applications/cilium/values.yaml` +file in your cluster management project. Refer to the [Cilium Helm reference](https://docs.cilium.io/en/stable/helm-reference/) +for the available configuration options. + +By default, Cilium's +[audit mode](https://docs.cilium.io/en/stable/gettingstarted/policy-creation/#enable-policy-audit-mode) +is enabled. In audit mode, Cilium doesn't drop disallowed packets. You +can use `policy-verdict` log to observe policy-related decisions. You +can disable audit mode by setting `policyAuditMode: false` in +`applications/cilium/values.yaml`. + +The Cilium monitor log for traffic is logged out by the +`cilium-monitor` sidecar container. You can check these logs with the following command: + +```shell +kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor +``` + +You can disable the monitor log in `application/cilium/values.yaml`: + +```yaml +monitor: + enabled: false +``` + +The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default +and it's set to collect per namespace flow metrics. This metrics are accessible on the +[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) +dashboard. You can disable Hubble by adding the following to +`applications/cilium/values.yaml`: + +```yaml +hubble: + enabled: false +``` + +You can also adjust Helm values for Hubble by using +`applications/cilium/values.yaml`: + +```yaml +hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' +``` + ## Managing Network Policies Managing NetworkPolicies through GitLab is advantageous over managing the policies in Kubernetes @@ -62,16 +156,14 @@ editor. To view statistics for Container Network Security, you must follow the installation steps above and configure GitLab integration with Prometheus. Also, if you use custom Helm values for Cilium, you must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Cilium values](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd): -your [Cilium values](../../../../clusters/applications.md#install-cilium-using-gitlab-cicd): +your [Cilium values](#use-the-cluster-management-template-to-install-cilium): ```yaml -global: - hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' +hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' ``` Additional information about the statistics page is available in the @@ -97,15 +189,14 @@ kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor By default, Cilium is installed in Audit mode only, meaning that NetworkPolicies log policy violations but don't block any traffic. To set Cilium to Blocking mode, you must add the following -lines to the `.gitlab/managed-apps/cilium/values.yaml` file in your cluster management project: +lines to the `applications/cilium/values.yaml` file in your cluster management project: ```yaml config: policyAuditMode: false -agent: - monitor: - eventTypes: ["drop"] +monitor: + eventTypes: ["drop"] ``` ### Traffic is not being allowed as expected @@ -120,7 +211,7 @@ traffic that you want to allow in the node. Occasionally, your CI/CD pipeline may fail or have trouble connecting to the cluster. Here are some initial troubleshooting steps that resolve the most common problems: -1. [Clear the cluster cache](../../index.md#clearing-the-cluster-cache). +1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache). 1. If things still aren't working, a more assertive set of actions may help get things back into a good state: diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 5d6fb8252bb..6eafb4530d3 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -134,7 +134,7 @@ This example code does the following: #### Setting up your AWS credentials with your GitLab account In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. -For more information please see [Create a custom variable in the UI](../../../../ci/variables/README.md#custom-variables-validated-by-gitlab). +For more information please see [Create a custom variable in the UI](../../../../ci/variables/index.md#custom-variables-validated-by-gitlab). The AWS credentials you provide must include IAM policies that provision correct access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. @@ -381,7 +381,7 @@ control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. ### Crafting the `.gitlab-ci.yml` file -In a [`.gitlab-ci.yml`](../../../../ci/yaml/README.md) file in the root of your project, +In a [`.gitlab-ci.yml`](../../../../ci/yaml/index.md) file in the root of your project, add the following and replace `<S3_bucket_name>` with the name of the S3 bucket where you want to store your package: diff --git a/doc/user/project/clusters/serverless/img/function-details-loaded.png b/doc/user/project/clusters/serverless/img/function-details-loaded.png Binary files differdeleted file mode 100644 index 2f0d61f8032..00000000000 --- a/doc/user/project/clusters/serverless/img/function-details-loaded.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png b/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png Binary files differnew file mode 100644 index 00000000000..a19d236fc39 --- /dev/null +++ b/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png diff --git a/doc/user/project/clusters/serverless/img/serverless-page.png b/doc/user/project/clusters/serverless/img/serverless-page.png Binary files differdeleted file mode 100644 index 8dce3cb1f70..00000000000 --- a/doc/user/project/clusters/serverless/img/serverless-page.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png b/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png Binary files differnew file mode 100644 index 00000000000..f88eb4bdcd2 --- /dev/null +++ b/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index e4ac1eabffe..ec22f71157f 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -322,7 +322,7 @@ being executed which deploys each function as a Knative service. After the deploy stage has finished, additional details for the function display under **Infrastructure > Serverless platform**. - + This page contains all functions available for the project, the description for accessing the function, and, if available, the function's runtime information. @@ -364,7 +364,7 @@ kubectl create secret generic my-secrets -n "$KUBE_NAMESPACE" --from-literal MY_ #### Part of deployment job -You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [CI/CD variables](../../../../ci/variables/README.md) +You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [CI/CD variables](../../../../ci/variables/index.md) stored securely under your GitLab project. ```yaml @@ -463,7 +463,7 @@ Go to the **Infrastructure > Serverless platform** page to see the final URL of On the same page as above, click on one of the function rows to bring up the function details page. - + The pod count gives you the number of pods running the serverless function instances on a given cluster. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index ea7bcce99c1..2a60c06814b 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -11,56 +11,56 @@ type: reference > - Code Owners for Merge Request approvals was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4418) in GitLab Premium 11.9. > - Moved to GitLab Premium in 13.9. -## Introduction +Code Owners define who owns specific files or paths in a repository. +You can require that Code Owners approve a merge request before it's merged. -When contributing to a project, it can often be difficult -to find out who should review or approve merge requests. -Additionally, if you have a question over a specific file or -code block, it may be difficult to know who to find the answer from. +Code Owners help you determine who should review or approve merge requests. +If you have a question about a file or feature, Code Owners +can help you find someone who knows the answer. -The GitLab Code Owners feature defines who owns specific -files or paths in a repository, allowing other users to understand -who is responsible for each file or path. +If you don't want to use Code Owners for approvals, you can +[configure rules](merge_requests/approvals/rules.md) instead. -As an alternative to using Code Owners for approvals, you can instead -[configure rules](merge_requests/approvals/rules.md). +## Set up Code Owners -## Why is this useful? +You can specify users or [shared groups](members/share_project_with_groups.md) +that are responsible for specific files and directories in a repository. -Code Owners allows for a version controlled, single source of -truth file outlining the exact GitLab users or groups that -own certain files or paths in a repository. In larger organizations -or popular open source projects, Code Owners can help you understand -who to contact if you have a question about a specific portion of -the codebase. Code Owners can also streamline the merge request approval -process, identifying the most relevant reviewers and approvers for a -given change. +To set up Code Owners: -## How to set up Code Owners +1. Choose the location where you want to specify Code Owners: + - In the root directory of the repository + - In the `.gitlab/` directory + - In the `docs/` directory -You can use a `CODEOWNERS` file to specify users or -[shared groups](members/share_project_with_groups.md) -that are responsible for specific files and directories in a repository. +1. In that location, create a file named `CODEOWNERS`. + +1. In the file, enter text that follows one of these patterns: + + ```plaintext + # A member as Code Owner of a file + filename @username + + # A member as Code Owner of a directory + directory @username -You can choose to add the `CODEOWNERS` file in three places: + # All group members as Code Owners of a file + filename @groupname -- To the root directory of the repository -- Inside the `.gitlab/` directory -- Inside the `docs/` directory + # All group members as Code Owners of a directory + directory @groupname + ``` -The `CODEOWNERS` file is valid for the branch where it lives. For example, if you change the code owners -in a feature branch, the changes aren't valid in the main branch until the feature branch is merged. +The Code Owners are displayed in the UI by the files or directory they apply to. +These owners apply to this branch only. When you add new files to the repository, +you should update the `CODEOWNERS` file. -If you introduce new files to your repository and you want to identify the code owners for that file, -you must update `CODEOWNERS` accordingly. If you update the code owners when you are adding the files (in the same -branch), GitLab counts the owners as soon as the branch is merged. If -you don't, you can do that later, but your new files don't belong to anyone until you update your -`CODEOWNERS` file in the TARGET branch. +## When a file matches multiple `CODEOWNERS` entries When a file matches multiple entries in the `CODEOWNERS` file, -the users from last pattern matching the file are displayed on the -blob page of the given file. For example, you have the following -`CODEOWNERS` file: +the users from last pattern matching the file are used. + +For example, in the following `CODEOWNERS` file: ```plaintext README.md @user1 @@ -77,7 +77,7 @@ After you've added Code Owners to a project, you can configure it to be used for merge request approvals: - As [merge request eligible approvers](merge_requests/approvals/rules.md#code-owners-as-eligible-approvers). -- As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners). **(PREMIUM)** +- As required approvers for [protected branches](protected_branches.md#require-code-owner-approval-on-a-protected-branch). **(PREMIUM)** Developer or higher [permissions](../permissions.md) are required to approve a merge request. @@ -93,11 +93,11 @@ without using [Approval Rules](merge_requests/approvals/rules.md): 1. Create the file in one of the three locations specified above. 1. Set the code owners as required approvers for - [protected branches](protected_branches.md#protected-branches-approval-by-code-owners). -1. Use [the syntax of Code Owners files](code_owners.md#the-syntax-of-code-owners-files) + [protected branches](protected_branches.md#require-code-owner-approval-on-a-protected-branch). +1. Use [the syntax of Code Owners files](code_owners.md) to specify the actual owners and granular permissions. -Using Code Owners in conjunction with [protected branches](protected_branches.md#protected-branches-approval-by-code-owners) +Using Code Owners in conjunction with [protected branches](protected_branches.md#require-code-owner-approval-on-a-protected-branch) prevents any user who is not specified in the `CODEOWNERS` file from pushing changes for the specified files/paths, except those included in the **Allowed to push** column. This allows for a more inclusive push strategy, as @@ -107,20 +107,7 @@ Code Owners is required. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5, users and groups who are allowed to push to protected branches do not require a merge request to merge their feature branches. Thus, they can skip merge request approval rules, Code Owners included. -## The syntax of Code Owners files - -Files can be specified using the same kind of patterns you would use -in the `.gitignore` file followed by one or more of: - -- A user's `@username`. -- A user's email address. -- The `@name` of one or more groups that should be owners of the file. -- Lines starting with `#` are ignored. - -The path definition order is significant: the last pattern -matching a given path is used to find the code owners. - -### Groups as Code Owners +## Groups as Code Owners > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53182) in GitLab 12.1. > - Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in GitLab 13.0. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 89c82d4dc6f..a09448d4755 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -59,7 +59,7 @@ specific environment, there are a lot of use cases. To name a few: - You want to promote what's running in staging, to production. You go to the environments list, verify that what's running in staging is what you think is - running, then click on the [manual action](../../ci/yaml/README.md#whenmanual) to deploy to production. + running, then click on the [manual action](../../ci/yaml/index.md#whenmanual) to deploy to production. - You trigger a deploy, and you have many containers to upgrade so you know this takes a while (you've also throttled your deploy to only take down X containers at a time). But you need to tell someone when it's deployed, so you @@ -88,7 +88,7 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ [OpenShift docs](https://docs.openshift.com/container-platform/3.7/dev_guide/deployments/kubernetes_deployments.html#kubernetes-deployments-vs-deployment-configurations) and [GitLab issue #4584](https://gitlab.com/gitlab-org/gitlab/-/issues/4584). -1. [Configure GitLab Runner](../../ci/runners/README.md) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +1. [Configure GitLab Runner](../../ci/runners/index.md) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/executors/kubernetes.html) executor. 1. Configure the [Kubernetes integration](clusters/index.md) in your project for the cluster. The Kubernetes namespace is of particular note as you need it @@ -165,6 +165,6 @@ version of your application. ## Further reading - [GitLab Auto deploy](../../topics/autodevops/stages.md#auto-deploy) -- [GitLab CI/CD variables](../../ci/variables/README.md) +- [GitLab CI/CD variables](../../ci/variables/index.md) - [Environments and deployments](../../ci/environments/index.md) - [Kubernetes deploy example](https://gitlab.com/gitlab-examples/kubernetes-deploy) diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index c0bc97781b6..1b9a17ee071 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -28,7 +28,7 @@ repository in automation, it's a simple solution. A drawback is that your repository could become vulnerable if a remote machine is compromised by a hacker. You should limit access to the remote machine before a deploy key is enabled on your repository. A good rule to follow is to provide access only to trusted users, -and make sure that the allowed users have the [Maintainer role or higher](../../permissions.md) +and make sure that the allowed users have at least the [Maintainer role](../../permissions.md) in the GitLab project. If this security implication is a concern for your organization, @@ -121,8 +121,9 @@ repositories to secure, shared services, such as CI/CD. Instance administrators can add public deploy keys: -1. Go to **Admin Area > Deploy Keys**. -1. Click on **New deploy key**. +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Deploy Keys**. +1. Select **New deploy key**. Make sure your new key has a meaningful title, as it is the primary way for project maintainers and owners to identify the correct public deploy key to add. For example, diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 711d7f561e4..72ef88b5fab 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -4,7 +4,7 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Description templates +# Description templates **(FREE)** We all know that a properly submitted issue is more likely to be addressed in a timely manner by the developers of a project. diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index eb963cb74c5..5ffde38b348 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -34,7 +34,7 @@ GitLab supports two different modes of file locking: ## Permissions Locks can be created by any person who has at least -[Developer permissions](../permissions.md) to the repository. +[Developer role](../permissions.md) in the repository. Only the user who locked the file or directory can edit locked files. Other users are prevented from modifying locked files by pushing, merging, diff --git a/doc/user/project/img/protected_branches_delete.png b/doc/user/project/img/protected_branches_delete.png Binary files differdeleted file mode 100644 index 8910ae9e39d..00000000000 --- a/doc/user/project/img/protected_branches_delete.png +++ /dev/null diff --git a/doc/user/project/img/protected_branches_deploy_keys_v13_5.png b/doc/user/project/img/protected_branches_deploy_keys_v13_5.png Binary files differdeleted file mode 100644 index ccd23dbe160..00000000000 --- a/doc/user/project/img/protected_branches_deploy_keys_v13_5.png +++ /dev/null diff --git a/doc/user/project/img/protected_branches_matches.png b/doc/user/project/img/protected_branches_matches.png Binary files differdeleted file mode 100644 index d7f2c8582fc..00000000000 --- a/doc/user/project/img/protected_branches_matches.png +++ /dev/null diff --git a/doc/user/project/img/protected_branches_select_roles_and_users.png b/doc/user/project/img/protected_branches_select_roles_and_users.png Binary files differdeleted file mode 100644 index 4f62b057cc7..00000000000 --- a/doc/user/project/img/protected_branches_select_roles_and_users.png +++ /dev/null diff --git a/doc/user/project/img/protected_branches_select_roles_and_users_list.png b/doc/user/project/img/protected_branches_select_roles_and_users_list.png Binary files differdeleted file mode 100644 index 519f2267496..00000000000 --- a/doc/user/project/img/protected_branches_select_roles_and_users_list.png +++ /dev/null diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index 963b9f524ff..7ccdb632c19 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -36,7 +36,7 @@ created as private in GitLab as well. - Attachments in Markdown are not imported. - Task lists are not imported. - Emoji reactions are not imported. -- Project filtering does not support fuzzy search (only `starts with` or `full match strings` are +- Project filtering does not support fuzzy search (only `starts with` or `full match strings` are supported). ## How it works @@ -51,7 +51,7 @@ The Bitbucket Server importer works as follows: ### User assignment When issues/pull requests are being imported, the Bitbucket importer tries to -find the author's e-mail address with a confirmed e-mail address in the GitLab +find the author's email address with a confirmed email address in the GitLab user database. If no such user is available, the project creator is set as the author. The importer appends a note in the comment to mark the original creator. diff --git a/doc/user/project/import/fogbugz.md b/doc/user/project/import/fogbugz.md index d3d77f16200..982bc6d90e8 100644 --- a/doc/user/project/import/fogbugz.md +++ b/doc/user/project/import/fogbugz.md @@ -7,31 +7,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import your project from FogBugz to GitLab **(FREE)** -It only takes a few simple steps to import your project from FogBugz. -The importer imports all your cases and comments with original case -numbers and timestamps. You can also map FogBugz users to GitLab users. +Using the importer, you can import your FogBugz project to GitLab.com +or to your self-managed GitLab instance. -Follow these steps to import your project from FogBugz: +The importer imports all of your cases and comments with the original +case numbers and timestamps. You can also map FogBugz users to GitLab +users. -1. From your GitLab dashboard, select **New project**. +To import your project from FogBugz: +1. From your GitLab dashboard, select **New project**. 1. Select the **FogBugz** button. - -  - +  1. Enter your FogBugz URL, email address, and password. - -  - -1. Create mapping from FogBugz users to GitLab users. - -  - -1. Select the projects you wish to import by selecting the **Import** buttons. - -  - -1. Once the import finishes, click the link to go to the project +  +1. Create a mapping from FogBugz users to GitLab users. +  +1. Select **Import** for the projects you want to import. +  +1. After the import finishes, click the link to go to the project dashboard. Follow the directions to push your existing repository. - -  +  diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 99b3e1acdcf..e67b6a45280 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -28,7 +28,7 @@ The following aspects of a project are imported: References to pull requests and issues are preserved (GitLab.com & 8.7+), and each imported repository maintains visibility level unless that [visibility -level is restricted](../../../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects), +level is restricted](../../../public_access/public_access.md#restrict-use-of-public-or-internal-projects), in which case it defaults to the default project visibility. The namespace is a user or group in GitLab, such as `gitlab.com/janedoe` or `gitlab.com/customer-success`. You can do some bulk actions to move projects to different namespaces in the rails console. @@ -60,7 +60,7 @@ For this association to succeed, each GitHub author and assignee in the reposito must meet one of the following conditions prior to the import: - Have previously logged in to a GitLab account using the GitHub icon. -- Have a GitHub account with a [public-facing email address](https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address) +- Have a GitHub account with a [public-facing email address](https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/managing-email-preferences/setting-your-commit-email-address) that matches their GitLab account's email address. NOTE: @@ -134,6 +134,9 @@ If you are not using the GitHub integration, you can still perform an authorizat 1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. Once done, you'll be taken to the importer page to select the repositories to import. +To use a newer personal access token in imports after previously performing these steps, sign out of +your GitLab account and sign in again, or revoke the older personal access token in GitHub. + ### Select which repositories to import After you have authorized access to your GitHub repositories, you are redirected to the GitHub importer page and @@ -160,6 +163,9 @@ Additionally, you can configure GitLab to send pipeline status updates back GitH If you import your project using [CI/CD for external repository](../../../ci/ci_cd_for_external_repos/index.md), then both of the above are automatically configured. **(PREMIUM)** +NOTE: +Mirroring does not sync any new or updated pull requests from your GitHub project. + ## Improve the speed of imports on self-managed instances NOTE: diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 05fd04f6e48..dcc41c6c85e 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -49,7 +49,7 @@ When migrating to GitLab.com, you must create users manually unless [SCIM](../.. will be used. Creating users with the API is limited to self-managed instances as it requires administrator access. -To migrate all data from self-managed to GitLab.com, you can leverage the [API](../../../api/README.md). +To migrate all data from self-managed to GitLab.com, you can leverage the [API](../../../api/index.md). Migrate the assets in this order: 1. [Groups](../../../api/groups.md) @@ -74,6 +74,10 @@ best to [back up](../../../raketasks/backup_restore.md) the existing instance and restore it on the new instance. For example, this is useful when migrating a self-managed instance from an old server to a new server. +The backups produced don't depend on the operating system running GitLab. You can therefore use +the restore method to switch between different operating system distributions or versions, as long +as the same GitLab version [is available for installation](https://docs.gitlab.com/omnibus/package-information/deprecated_os.md). + To instead merge two self-managed GitLab instances together, use the instructions in [Migrate from self-managed GitLab to GitLab.com](#migrate-from-self-managed-gitlab-to-gitlabcom). This method is useful when both self-managed instances have existing data that must be preserved. diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index 4273f90c1e7..07419080d7d 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -4,7 +4,7 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Import your Jira project issues to GitLab +# Import your Jira project issues to GitLab **(PREMIUM)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2766) in GitLab 12.10. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 0dcbf997452..fca9c3e7023 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -65,7 +65,7 @@ Projects include the following [features](https://about.gitlab.com/features/): **GitLab CI/CD:** -- [GitLab CI/CD](../../ci/README.md): Use the built-in [Continuous Integration, Delivery, and Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) tool. +- [GitLab CI/CD](../../ci/index.md): Use the built-in [Continuous Integration, Delivery, and Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) tool. - [Container Registry](../packages/container_registry/index.md): Build and push Docker images. - [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy): Configure GitLab CI/CD @@ -134,7 +134,7 @@ To review or contribute to the extension's code, visit [its codebase in GitLab]( ## Project APIs -There are numerous [APIs](../../api/README.md) to use with your projects: +There are numerous [APIs](../../api/index.md) to use with your projects: - [Badges](../../api/project_badges.md) - [Clusters](../../api/project_clusters.md) diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index 4f5640d9fff..019ca9da9f1 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -18,7 +18,7 @@ and is automatically configured on [GitHub import](../../../integration/github.m ## Configuration -This integration requires a [GitHub API token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) +This integration requires a [GitHub API token](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) with `repo:status` access granted. Complete these steps on GitHub: diff --git a/doc/user/project/integrations/hangouts_chat.md b/doc/user/project/integrations/hangouts_chat.md index 0668e5dd88f..d5dc02d5455 100644 --- a/doc/user/project/integrations/hangouts_chat.md +++ b/doc/user/project/integrations/hangouts_chat.md @@ -36,7 +36,7 @@ Select a room and create a webhook: 1. Select **Save**. 1. Copy the webhook URL. -For further details, see [the Google Chat documentation for configuring webhooks](https://developers.google.com/hangouts/chat/how-tos/webhooks). +For further details, see [the Google Chat documentation for configuring webhooks](https://developers.google.com/chat/how-tos/webhooks). ## In GitLab diff --git a/doc/user/project/integrations/img/project_integrations_v13_3.png b/doc/user/project/integrations/img/project_integrations_v13_3.png Binary files differdeleted file mode 100644 index 9c925d32441..00000000000 --- a/doc/user/project/integrations/img/project_integrations_v13_3.png +++ /dev/null diff --git a/doc/user/project/integrations/jira_cloud_configuration.md b/doc/user/project/integrations/jira_cloud_configuration.md deleted file mode 100644 index c9ab4532760..00000000000 --- a/doc/user/project/integrations/jira_cloud_configuration.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../../integration/jira/jira_cloud_configuration.md' -remove_date: '2021-06-18' ---- - -This document was moved to [another location](../../../integration/jira/jira_cloud_configuration.md). - -<!-- This redirect file can be deleted after <2021-06-18>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md deleted file mode 100644 index de6eec62b96..00000000000 --- a/doc/user/project/integrations/jira_server_configuration.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../../integration/jira/jira_server_configuration.md' -remove_date: '2021-06-18' ---- - -This document was moved to [another location](../../../integration/jira/jira_server_configuration.md). - -<!-- This redirect file can be deleted after <2021-06-18>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index 834bf15c287..619ae52481b 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -141,7 +141,7 @@ The available slash commands for Mattermost are: | ------- | ----------- | ------- | | <kbd>/<trigger> issue new <title> <kbd>⇧ Shift</kbd>+<kbd>↵ Enter</kbd> <description></kbd> | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | `/gitlab issue new We need to change the homepage` | | <kbd>/<trigger> issue show <issue-number></kbd> | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | `/gitlab issue show 42` | -| <kbd>/<trigger> deploy <environment> to <environment></kbd> | Start the CI job that deploys from one environment to another, for example `staging` to `production`. CI/CD must be [properly configured](../../../ci/yaml/README.md). | `/gitlab deploy staging to production` | +| <kbd>/<trigger> deploy <environment> to <environment></kbd> | Start the CI job that deploys from one environment to another, for example `staging` to `production`. CI/CD must be [properly configured](../../../ci/yaml/index.md). | `/gitlab deploy staging to production` | To see a list of available commands to interact with GitLab, type the trigger word followed by <kbd>help</kbd>. Example: `/gitlab help` diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index b0ae290e7cd..53aa9da30ab 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -15,11 +15,9 @@ functionality to GitLab. You can find the available integrations under your project's **Settings > Integrations** page. -There are more than 20 integrations to integrate with. Click on the one that you +There are more than 20 integrations to integrate with. Select the one that you want to configure. - - ## Integrations listing Click on the service links to see further configuration instructions and details. @@ -34,6 +32,7 @@ Click on the service links to see further configuration instructions and details | Campfire | Connect to chat. | **{dotted-circle}** No | | [Confluence Workspace](../../../api/services.md#confluence-service) | Replace the link to the internal wiki with a link to a Confluence Cloud Workspace. | **{dotted-circle}** No | | [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | +| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | | [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | | Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | | [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | @@ -51,7 +50,7 @@ Click on the service links to see further configuration instructions and details | [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | | Packagist | Update your projects. | **{check-circle}** Yes | | Pipelines emails | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | -| PivotalTracker | Use PivotalTracker as the issue tracker. | **{dotted-circle}** No | +| [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | | [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | | Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | | [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | @@ -115,6 +114,6 @@ plugins. This allows the community to keep the plugins up to date so that they always work in newer GitLab versions. For an overview of what integrations are available, please see the -[project_services source directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/models/project_services). +[integrations source directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/models/integrations). Contributions are welcome! diff --git a/doc/user/project/integrations/pivotal_tracker.md b/doc/user/project/integrations/pivotal_tracker.md new file mode 100644 index 00000000000..c2c827c240b --- /dev/null +++ b/doc/user/project/integrations/pivotal_tracker.md @@ -0,0 +1,47 @@ +--- +stage: Create +group: Ecosystem +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Pivotal Tracker service **(FREE)** + +This service adds commit messages as comments to Pivotal Tracker stories. + +Once enabled, commit messages are checked for square brackets containing a hash mark followed by +the story ID (for example, `[#555]`). Every story ID found gets the commit comment added to it. + +You can also close a story with a message containing: `fix [#555]`. +You can use any of these words: + +- `fix` +- `fixed` +- `fixes` +- `complete` +- `completes` +- `completed` +- `finish` +- `finished` +- `finishes` +- `delivers` + +Read more about the +[Source Commits endpoint](https://www.pivotaltracker.com/help/api/rest/v5#Source_Commits) in +the Pivotal Tracker API documentation. + +See also the [Pivotal Tracker service API documentation](../../../api/services.md#pivotal-tracker). + +## Set up Pivotal Tracker + +In Pivotal Tracker, [create an API token](https://www.pivotaltracker.com/help/articles/api_token/). + +Complete these steps in GitLab: + +1. Go to the project you want to configure. +1. Go to the [Integrations page](overview.md#accessing-integrations). +1. Select **Pivotal Tracker**. +1. Ensure that the **Active** toggle is enabled. +1. Paste the token you generated in Pivotal Tracker. +1. (Optional) To restrict this setting to specific branches, list them in the **Restrict to branch** + field, separated with commas. +1. Select **Save changes** or optionally select **Test settings**. diff --git a/doc/user/project/integrations/prometheus_library/index.md b/doc/user/project/integrations/prometheus_library/index.md index 584c0898fec..fe74ea6834b 100644 --- a/doc/user/project/integrations/prometheus_library/index.md +++ b/doc/user/project/integrations/prometheus_library/index.md @@ -35,6 +35,6 @@ In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do that, GitLab uses the defined queries and fills in the environment specific variables. Typically this involves looking for the -[`$CI_ENVIRONMENT_SLUG`](../../../../ci/variables/README.md#predefined-cicd-variables), +[`$CI_ENVIRONMENT_SLUG`](../../../../ci/variables/index.md#predefined-cicd-variables), but may also include other information such as the project's Kubernetes namespace. Each search query is defined in the [exporter specific documentation](#exporters). diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 1bafa4938af..ea0119f2e94 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -40,7 +40,7 @@ Prometheus needs to be deployed into the cluster and configured properly in orde In order to isolate and only display relevant CPU and Memory metrics for a given environment, GitLab needs a method to detect which containers it is running. Because these metrics are tracked at the container level, traditional Kubernetes labels are not available. -Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name should begin with [CI_ENVIRONMENT_SLUG](../../../../ci/variables/README.md#predefined-cicd-variables). It can be followed by a `-` and additional content if desired. For example, a deployment name of `review-homepage-5620p5` would match the `review/homepage` environment. +Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name should begin with [CI_ENVIRONMENT_SLUG](../../../../ci/variables/index.md#predefined-cicd-variables). It can be followed by a `-` and additional content if desired. For example, a deployment name of `review-homepage-5620p5` would match the `review/homepage` environment. ## Displaying Canary metrics **(PREMIUM)** diff --git a/doc/user/project/integrations/webex_teams.md b/doc/user/project/integrations/webex_teams.md index 05515c58161..2851fe0b299 100644 --- a/doc/user/project/integrations/webex_teams.md +++ b/doc/user/project/integrations/webex_teams.md @@ -13,7 +13,7 @@ You can configure GitLab to send notifications to a Webex Teams space: ## Create a webhook for the space -1. Go to the [Incoming Webhooks app page](https://apphub.webex.com/messaging/applications/incoming-webhooks-cisco-systems-38054). +1. Go to the [Incoming Webhooks app page](https://apphub.webex.com/applications/incoming-webhooks-cisco-systems-38054). 1. Select **Connect** and log in to Webex Teams, if required. 1. Enter a name for the webhook and select the space to receive the notifications. 1. Select **ADD**. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 406b1e9ba6b..01f3424d993 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -112,7 +112,7 @@ branches, this hook isn't executed. X-Gitlab-Event: Push Hook ``` -**Request body:** +**Payload example:** ```json { @@ -202,7 +202,7 @@ tags, this hook is not executed. X-Gitlab-Event: Tag Push Hook ``` -**Request body:** +**Payload example:** ```json { @@ -256,7 +256,14 @@ Triggered when a new issue is created or an existing issue was updated/closed/re X-Gitlab-Event: Issue Hook ``` -**Request body:** +**Available `object_attributes.action`:** + +- `open` +- `close` +- `reopen` +- `update` + +**Payload example:** ```json { @@ -423,7 +430,7 @@ Valid target types: X-Gitlab-Event: Note Hook ``` -**Request body:** +**Payload example:** ```json { @@ -505,7 +512,7 @@ X-Gitlab-Event: Note Hook X-Gitlab-Event: Note Hook ``` -**Request body:** +**Payload example:** ```json { @@ -634,7 +641,7 @@ X-Gitlab-Event: Note Hook X-Gitlab-Event: Note Hook ``` -**Request body:** +**Payload example:** ```json { @@ -742,7 +749,7 @@ NOTE: X-Gitlab-Event: Note Hook ``` -**Request body:** +**Payload example:** ```json { @@ -820,7 +827,17 @@ Triggered when a new merge request is created, an existing merge request was upd X-Gitlab-Event: Merge Request Hook ``` -**Request body:** +**Available `object_attributes.action`:** + +- `open` +- `close` +- `reopen` +- `update` +- `approved` +- `unapproved` +- `merge` + +**Payload example:** ```json { @@ -983,7 +1000,7 @@ Triggered when a wiki page is created, updated or deleted. X-Gitlab-Event: Wiki Page Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1044,7 +1061,7 @@ Triggered on status change of Pipeline. X-Gitlab-Event: Pipeline Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1291,7 +1308,7 @@ Triggered on status change of a job. X-Gitlab-Event: Job Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1340,7 +1357,7 @@ X-Gitlab-Event: Job Hook }, "runner": { "active": true, - "runner_type": "project_type", + "runner_type": "project_type", "is_shared": false, "id": 380987, "description": "shared-runners-manager-6.gitlab.com", @@ -1370,7 +1387,7 @@ Triggered when a deployment: X-Gitlab-Event: Deployment Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1433,7 +1450,7 @@ Member events are triggered when: X-Gitlab-Event: Member Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1461,7 +1478,7 @@ X-Gitlab-Event: Member Hook X-Gitlab-Event: Member Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1489,7 +1506,7 @@ X-Gitlab-Event: Member Hook X-Gitlab-Event: Member Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1526,7 +1543,7 @@ Subgroup events are triggered when: X-Gitlab-Event: Subgroup Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1554,7 +1571,7 @@ X-Gitlab-Event: Subgroup Hook X-Gitlab-Event: Subgroup Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1587,7 +1604,7 @@ Triggered when a feature flag is turned on or off. X-Gitlab-Event: Feature Flag Hook ``` -**Request Body**: +**Payload example**: ```json { @@ -1637,7 +1654,12 @@ Triggered when a release is created or updated. X-Gitlab-Event: Release Hook ``` -**Request Body**: +**Available `object_attributes.action`:** + +- `create` +- `update` + +**Payload example**: ```json { @@ -1762,6 +1784,10 @@ On this page, you can see data that GitLab sends (request headers and body) and From this page, you can repeat delivery with the same data by clicking `Resend Request` button. NOTE: +This history is unavailable for Group-level webhooks. For more information, read +[issue #325642](https://gitlab.com/gitlab-org/gitlab/-/issues/325642). + +NOTE: If URL or secret token of the webhook were updated, data is delivered to the new address. ### Webhook fails or multiple webhook requests are triggered diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 8f71d469e34..a32a8ed8ec7 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -97,8 +97,8 @@ For examples of using issue boards along with [epics](../group/epics/index.md), ### Use cases for a single issue board -With the GitLab Workflow you can discuss proposals in issues, label -them, and organize and prioritize them with issue boards. +With the [GitLab Flow](https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/) you can +discuss proposals in issues, label them, and organize and prioritize them with issue boards. For example, let's consider this simplified development workflow: @@ -154,7 +154,7 @@ for them. NOTE: For a broader use case, please see the blog post -[GitLab Workflow, an Overview](https://about.gitlab.com/topics/version-control/what-is-gitlab-workflow/#gitlab-workflow-a-use-case-scenario). +[What is GitLab Flow?](https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/). For a real use case example, you can read why [Codepen decided to adopt issue boards](https://about.gitlab.com/blog/2017/01/27/codepen-welcome-to-gitlab/#project-management-everything-in-one-place) to improve their workflow with multiple boards. @@ -426,7 +426,7 @@ To set a WIP limit for a list: 1. Enter the maximum number of issues. 1. Press <kbd>Enter</kbd> to save. -## Blocked issues +## Blocked issues **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34723) in GitLab 12.8. > - [View blocking issues when hovering over blocked icon](https://gitlab.com/gitlab-org/gitlab/-/issues/210452) in GitLab 13.10. diff --git a/doc/user/project/issues/associate_zoom_meeting.md b/doc/user/project/issues/associate_zoom_meeting.md index f98e94c66ae..e020bdee737 100644 --- a/doc/user/project/issues/associate_zoom_meeting.md +++ b/doc/user/project/issues/associate_zoom_meeting.md @@ -4,7 +4,7 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Associate a Zoom meeting with an issue +# Associate a Zoom meeting with an issue **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4. diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index ed15d7a2e63..92c26fb654e 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -4,11 +4,9 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Confidential issues +# Confidential issues **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3282) in GitLab 8.6. - -Confidential issues are issues visible only to members of a project with +Confidential issues are [issues](index.md) visible only to members of a project with [sufficient permissions](#permissions-and-access-to-confidential-issues). Confidential issues can be used by open source projects and companies alike to keep security vulnerabilities private or prevent surprises from leaking out. diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index 63b38520c98..2b07131df6e 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -4,9 +4,9 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Crosslinking issues +# Crosslinking issues **(FREE)** -There are several ways to mention an issue or make issues appear in each other's +There are several ways to mention an issue or make [issues](index.md) appear in each other's [Linked issues](related_issues.md) section. For more information on GitLab Issues, read the [issues documentation](index.md). diff --git a/doc/user/project/issues/img/issue_type_change_v13_12.png b/doc/user/project/issues/img/issue_type_change_v13_12.png Binary files differindex 3b4864ffbbb..55aa607b878 100644 --- a/doc/user/project/issues/img/issue_type_change_v13_12.png +++ b/doc/user/project/issues/img/issue_type_change_v13_12.png diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 13f5beadb16..2ef12cd1240 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -4,7 +4,7 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Issue Data and Actions +# Issue Data and Actions **(FREE)** Please read through the [GitLab Issue Documentation](index.md) for an overview on GitLab Issues. @@ -151,7 +151,7 @@ cannot access the issue, and it is not listed in the project's issue boards nor ### Lock issue -You can [lock the threads](../../discussions/index.md#lock-discussions) in the issue, +You can [lock the issue](../../discussions/index.md#prevent-comments-by-locking-an-issue) to prevent further comments from being added. ### Participants @@ -288,7 +288,7 @@ supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown). After you write a comment, you can: - Click **Comment** to publish your comment. -- Choose **Start thread** from the dropdown list and start a new [thread](../../discussions/index.md#threaded-discussions) +- Choose **Start thread** from the dropdown list and start a new [thread](../../discussions/index.md#create-a-thread-without-replying-to-a-comment) in that issue's main thread to discuss specific points. This invites other participants to reply directly to your thread, keeping related comments grouped together. diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 35573518626..c570bc9612a 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -4,7 +4,7 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Managing issues +# Managing issues **(FREE)** [GitLab Issues](index.md) are the fundamental medium for collaborating on ideas and planning work in GitLab. @@ -333,7 +333,7 @@ of your installation. ## Change the issue type -Users with [developer permission](../../permissions.md) +Users with the [Developer role](../../permissions.md) can change an issue's type. To do this, edit the issue and select an issue type from the **Issue type** selector menu: diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index 97a790c2527..2681a39aeb6 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can sort a list of issues several ways, including by: -- Blocking +- Blocking **(PREMIUM)** - Created date - Due date - Label priority @@ -51,7 +51,7 @@ This ordering also affects [issue boards](../issue_board.md#how-gitlab-orders-is Changing the order in an issue list changes the ordering in an issue board, and vice versa. -## Sorting by blocking issues +## Sorting by blocking issues **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34247/) in GitLab 13.7. diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index e7adc045e98..43d6ab2070d 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -59,7 +59,7 @@ and edit labels. > Showing all inherited labels [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241990) in GitLab 13.5. -To view a project's available labels, in the project, go to **Issues > Labels**. +To view a project's available labels, in the project, go to **Project information > Labels**. Its list of labels includes both the labels defined at the project level, and all labels defined by its ancestor groups. For each label, you can see the project or group path from where it was created. You can filter the list by @@ -68,7 +68,7 @@ icon (**{search}**). To create a new project label: -1. In your project, go to **Issues > Labels**. +1. In your project, go to **Project information > Labels**. 1. Select the **New label** button. 1. In the **Title** field, enter a short, descriptive name for the label. You can also use this field to create [scoped, mutually exclusive labels](#scoped-labels). @@ -118,18 +118,18 @@ Promoting a label is a permanent action, and cannot be reversed. To promote a project label to a group label: -1. Navigate to **Issues > Labels** in the project. +1. Navigate to **Project information > Labels** in the project. 1. Click on the three dots (**{ellipsis_v}**) next to the **Subscribe** button and select **Promote to group label**. ### Group labels -To view the group labels list, navigate to the group and click **Issues > Labels**. +To view the group labels list, navigate to the group and click **Group information > Labels**. The list includes all labels that are defined at the group level only. It does not list any labels that are defined in projects. You can filter the list by entering a search query at the top and clicking search (**{search}**). -To create a **group label**, navigate to **Issues > Labels** in the group and +To create a **group label**, navigate to **Group information > Labels** in the group and follow the same process as [creating a project label](#project-labels). #### Create group labels from epics **(ULTIMATE)** diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 11d6bfb5d0c..8987c663860 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -24,7 +24,7 @@ To add a user to a project: 1. Go to your project and select **Project information > Members**. 1. On the **Invite member** tab, under **GitLab member or Email address**, type the username or email address. In GitLab 13.11 and later, you can [replace this form with a modal window](#add-a-member-modal-window). -1. Select a [role](../../permissions.md). +1. Select a [role](../../permissions.md). 1. Optional. Choose an expiration date. On that date, the user can no longer access the project. 1. Select **Invite**. @@ -54,7 +54,7 @@ To add groups to a project: 1. Go to your project and select **Project information > Members**. 1. On the **Invite group** tab, under **Select a group to invite**, choose a group. -1. Select the highest max [role](../../permissions.md) for users in the group. +1. Select the highest max [role](../../permissions.md) for users in the group. 1. Optional. Choose an expiration date. On that date, the user can no longer access the project. 1. Select **Invite**. @@ -99,6 +99,8 @@ In this example: - **Administrator** is the [Owner](../../permissions.md) and member of all groups. They have inherited their role from the **demo** group. +If a user is a direct member of a project, the expiration date can be updated. If membership is inherited from a parent group, the expiration date can be updated only from the parent group itself. + ## Remove a member from a project If a user is a direct member of a project, you can remove them. @@ -156,7 +158,7 @@ You can sort members by **Account**, **Access granted**, **Max role**, or **Last ## Request access to a project -GitLab users can request to become a member of a project. +GitLab users can request to become a member of a project. 1. Go to the project you'd like to be a member of. 1. By the project name, select **Request Access**. diff --git a/doc/user/project/merge_requests/accessibility_testing.md b/doc/user/project/merge_requests/accessibility_testing.md index 76aff18b00d..2bc6d5bb148 100644 --- a/doc/user/project/merge_requests/accessibility_testing.md +++ b/doc/user/project/merge_requests/accessibility_testing.md @@ -10,7 +10,7 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25144) in GitLab 12.8. If your application offers a web interface and you are using -[GitLab CI/CD](../../../ci/README.md), you can quickly determine the accessibility +[GitLab CI/CD](../../../ci/index.md), you can quickly determine the accessibility impact of pending code changes. ## Overview @@ -37,7 +37,7 @@ This example shows how to run [pa11y](https://pa11y.org/) on your code with GitLab CI/CD using the [GitLab Accessibility Docker image](https://gitlab.com/gitlab-org/ci-cd/accessibility). For GitLab 12.9 and later, to define the `a11y` job, you must -[include](../../../ci/yaml/README.md#includetemplate) the +[include](../../../ci/yaml/index.md#includetemplate) the [`Accessibility.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml) included with your GitLab installation, as shown below. diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index 3c47c2af344..40345f33cb2 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -11,11 +11,21 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/merge > Redesign [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1979) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/10685) in 12.0. You can configure your merge requests so that they must be approved before -they can be merged. You can do this by creating [rules](rules.md) or by specifying -a list of users who act as [code owners](../../code_owners.md) for specific files. - -You can configure merge request approvals for each project. In higher GitLab tiers, -Administrators of self-managed GitLab instances can configure approvals +they can be merged. While [GitLab Free](https://about.gitlab.com/pricing/) allows +all users with Developer or greater [permissions](../../../permissions.md) to +approve merge requests, these approvals are [optional](#optional-approvals). +[GitLab Premium](https://about.gitlab.com/pricing/) and +[GitLab Ultimate](https://about.gitlab.com/pricing/) provide additional +flexibility: + +- Create required [rules](rules.md) about the number and type of approvers before work can merge. +- Specify a list of users who act as [code owners](../../code_owners.md) for specific files, + and require their approval before work can merge. + +You can configure merge request approvals on a per-project basis. Administrators of +[GitLab Premium](https://about.gitlab.com/pricing/) and +[GitLab Ultimate](https://about.gitlab.com/pricing/) self-managed GitLab instances +can also configure approvals [for the entire instance](../../../admin_area/merge_requests_approvals.md). ## How approvals work @@ -60,7 +70,7 @@ a merge request. After a merge request receives the [number and type of approvals](rules.md) you configure, it can merge unless it's blocked for another reason. Merge requests can be blocked by other problems, -such as merge conflicts, [pending discussions](../../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-threads-are-resolved), +such as merge conflicts, [pending discussions](../../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved), or a [failed CI/CD pipeline](../merge_when_pipeline_succeeds.md). To prevent merge request authors from approving their own merge requests, @@ -94,6 +104,7 @@ Without the approvals, the work cannot merge. Required approvals enable multiple database, for all proposed code changes. - Use the [code owners of changed files](rules.md#code-owners-as-eligible-approvers), to determine who should review the work. +- Require an [approval before merging code that causes test coverage to decline](../../../../ci/pipelines/settings.md#coverage-check-approval-rule) - [Require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) before merging code that could introduce a vulnerability. **(ULTIMATE)** diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index 1e4b0f659ee..82685f9101e 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, concepts --- -# Merge request approval rules **(FREE)** +# Merge request approval rules **(PREMIUM)** Approval rules define how many [approvals](index.md) a merge request must receive before it can be merged, and which users should do the approving. You can define approval rules: @@ -159,7 +159,7 @@ become eligible approvers in the project. To enable this merge request approval  You can also -[require code owner approval](../../protected_branches.md#protected-branches-approval-by-code-owners) +[require code owner approval](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch) for protected branches. **(PREMIUM)** ## Merge request approval segregation of duties **(PREMIUM)** @@ -173,6 +173,7 @@ Some users (like managers) may not need permission to push or merge code, but st oversight on proposed work. To enable approval permissions for these users without granting them push access: +1. [Create a protected branch](../../protected_branches.md) 1. [Create a new group](../../../group/index.md#create-a-group). 1. [Add the user to the group](../../../group/index.md#add-users-to-a-group), and select the Reporter role for the user. @@ -180,7 +181,7 @@ granting them push access: based on the Reporter role. 1. Go to your project and select **Settings > General**. 1. Expand **Merge request (MR) approvals**. -1. Select **Add approval rule** or **Update approval rule**. +1. Select **Add approval rule** or **Update approval rule** and target the protected branch. 1. [Add the group](../../../group/index.md#create-a-group) to the permission list.  @@ -203,7 +204,7 @@ on a merge request, you can either add or remove approvers: Administrators can change the [merge request approvals settings](settings.md#prevent-overrides-of-default-approvals) to prevent users from overriding approval rules for merge requests. -## Configure optional approval rules +## Configure optional approval rules **(PREMIUM)** Merge request approvals can be optional for projects where approvals are appreciated, but not required. To make an approval rule optional: @@ -229,4 +230,4 @@ approval rule for certain branches:  1. To enable this configuration, read - [Code Owner's approvals for protected branches](../../protected_branches.md#protected-branches-approval-by-code-owners). + [Code Owner's approvals for protected branches](../../protected_branches.md#require-code-owner-approval-on-a-protected-branch). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index b72a4125d0e..8a81ff8c94b 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -115,8 +115,16 @@ permission enables an electronic signature for approvals, such as the one define ## Security approvals in merge requests **(ULTIMATE)** You can require that a member of your security team approves a merge request if a -merge request could introduce a vulnerability. To learn more, see -[Security approvals in merge requests](../../../application_security/index.md#security-approvals-in-merge-requests). +merge request could introduce a vulnerability. + +To learn more, see [Security approvals in merge requests](../../../application_security/index.md#security-approvals-in-merge-requests). + +## Code coverage check approvals **(PREMIUM)** + +You can require specific approvals if a merge request would result in a decline in code test +coverage. + +To learn more, see [Coverage check approval rule](../../../../ci/pipelines/settings.md#coverage-check-approval-rule). ## Related links diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md index 930df65f3fc..339f67f828f 100644 --- a/doc/user/project/merge_requests/authorization_for_merge_requests.md +++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md @@ -17,7 +17,7 @@ There are two main ways to have a merge request flow with GitLab: With the protected branch flow everybody works within the same GitLab project. The project maintainers get the [Maintainer role](../../permissions.md) and the regular developers -get Developer access. +get the Developer role. Maintainers mark the authoritative branches as 'Protected'. diff --git a/doc/user/project/merge_requests/browser_performance_testing.md b/doc/user/project/merge_requests/browser_performance_testing.md index d11ad53a9d6..eff3a5bd99e 100644 --- a/doc/user/project/merge_requests/browser_performance_testing.md +++ b/doc/user/project/merge_requests/browser_performance_testing.md @@ -10,7 +10,7 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. If your application offers a web interface and you're using -[GitLab CI/CD](../../../ci/README.md), you can quickly determine the rendering performance +[GitLab CI/CD](../../../ci/index.md), you can quickly determine the rendering performance impact of pending code changes in the browser. NOTE: @@ -40,7 +40,7 @@ Consider the following workflow: ## How browser performance testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Browser Performance report artifact](../../../ci/yaml/README.md#artifactsreportsbrowser_performance). +[Browser Performance report artifact](../../../ci/yaml/index.md#artifactsreportsbrowser_performance). GitLab then checks this report, compares key performance metrics for each page between the source and target branches, and shows the information in the merge request. @@ -89,7 +89,7 @@ The above example: GitLab 12.3 or earlier, you must [add the configuration manually](#gitlab-versions-132-and-earlier). The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance), -and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/yaml/README.md#artifactsreportsbrowser_performance) +and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/yaml/index.md#artifactsreportsbrowser_performance) that you can later download and analyze. This implementation always takes the latest Browser Performance artifact available. If [GitLab Pages](../pages/index.md) is enabled, you can view the report directly in your browser. diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 27642a9bd5d..19302572dc9 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -10,17 +10,22 @@ type: reference, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in GitLab 9.3. > - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in 13.2. -Ensuring your project's code stays simple, readable and easy to contribute to can be problematic. With the help of [GitLab CI/CD](../../../ci/README.md), you can analyze your -source code quality using GitLab Code Quality. +To ensure your project's code stays simple, readable, and easy to contribute to, +you can use [GitLab CI/CD](../../../ci/index.md) to analyze your source code quality. + +For example, while you're implementing a feature, you can run Code Quality reports +to analyze how your improvements are impacting your code's quality. You can +use this information to ensure that your changes are improving performance rather +than degrading it. Code Quality: -- Uses [Engines](https://docs.codeclimate.com/docs/list-of-engines) supported by Code Climate, which are +- Uses [plugins](https://docs.codeclimate.com/docs/list-of-engines) supported by Code Climate, which are free and open source. Code Quality does not require a Code Climate subscription. -- Runs in [pipelines](../../../ci/pipelines/index.md) using a Docker image built in the - [GitLab Code - Quality](https://gitlab.com/gitlab-org/ci-cd/codequality) project using [default Code Climate configurations](https://gitlab.com/gitlab-org/ci-cd/codequality/-/tree/master/codeclimate_defaults). +- Runs in [pipelines](../../../ci/pipelines/index.md) by using a Docker image built in the + [GitLab Code Quality](https://gitlab.com/gitlab-org/ci-cd/codequality) project. +- Uses [default Code Climate configurations](https://gitlab.com/gitlab-org/ci-cd/codequality/-/tree/master/codeclimate_defaults). - Can make use of a [template](#example-configuration). - Is available by using [Auto Code Quality](../../../topics/autodevops/stages.md#auto-code-quality), provided by [Auto DevOps](../../../topics/autodevops/index.md). - Can be extended through [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) or a [custom tool](#implementing-a-custom-tool). @@ -54,42 +59,14 @@ See also the Code Climate list of [Supported Languages for Maintainability](http > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267612) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.11. > - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/284140) in GitLab 13.12. -> - [Feature enhanced](https://gitlab.com/gitlab-org/gitlab/-/issues/2526) in GitLab 14.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284140) in GitLab 14.1. +> - [Inline annotation added](https://gitlab.com/gitlab-org/gitlab/-/issues/2526) in GitLab 14.1. Changes to files in merge requests can cause Code Quality to fall if merged. In these cases, the merge request's diff view displays an indicator next to lines with new Code Quality violations. For example:  -Previously, an indicator was displayed (**{information-o}** **Code Quality**) on the file in the merge request's diff view: - - - -To switch to the previous version of this feature, a GitLab administrator can run the following in a -[Rails console](../../../administration/operations/rails_console.md): - -```ruby -# For the instance -Feature.disable(:codequality_mr_diff_annotations) -# For a single project -Feature.disable(:codequality_mr_diff_annotations, Project.find(<project id>)) -``` - -## Use cases - -For instance, consider the following workflow: - -1. Your backend team member starts a new implementation for making a certain - feature in your app faster. -1. With Code Quality reports, they analyze how their implementation is impacting - the code quality. -1. The metrics show that their code degrades the quality by 10 points. -1. You ask a co-worker to help them with this modification. -1. They both work on the changes until Code Quality report displays no - degradations, only improvements. -1. You approve the merge request and authorize its deployment to staging. -1. Once verified, their changes are deployed to production. - ## Example configuration This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. @@ -111,7 +88,7 @@ include: The above example creates a `code_quality` job in your CI/CD pipeline which scans your source code for code quality issues. The report is saved as a -[Code Quality report artifact](../../../ci/yaml/README.md#artifactsreportscodequality) +[Code Quality report artifact](../../../ci/yaml/index.md#artifactsreportscodequality) that you can later download and analyze. It's also possible to override the URL to the Code Quality image by @@ -262,13 +239,13 @@ was chosen as an operational decision by the runner team, instead of exposing `d ### Disabling the code quality job The `code_quality` job doesn't run if the `$CODE_QUALITY_DISABLED` CI/CD variable -is present. Please refer to the CI/CD variables [documentation](../../../ci/variables/README.md) +is present. Please refer to the CI/CD variables [documentation](../../../ci/variables/index.md) to learn more about how to define one. To disable the `code_quality` job, add `CODE_QUALITY_DISABLED` as a custom CI/CD variable. This can be done: -- For [the whole project](../../../ci/variables/README.md#custom-cicd-variables). +- For [the whole project](../../../ci/variables/index.md#custom-cicd-variables). - For a single pipeline run: 1. Go to **CI/CD > Pipelines** @@ -278,11 +255,11 @@ This can be done: ### Using with merge request pipelines The configuration provided by the Code Quality template does not let the `code_quality` job -run on [pipelines for merge requests](../../../ci/merge_request_pipelines/index.md). +run on [pipelines for merge requests](../../../ci/pipelines/merge_request_pipelines.md). If pipelines for merge requests is enabled, the `code_quality:rules` must be redefined. -The template has these [`rules`](../../../ci/yaml/README.md#rules) for the `code quality` job: +The template has these [`rules`](../../../ci/yaml/index.md#rules) for the `code quality` job: ```yaml code_quality: @@ -292,7 +269,7 @@ code_quality: - if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH' ``` -If you are using merge request pipelines, your `rules` (or [`workflow: rules`](../../../ci/yaml/README.md#workflow)) +If you are using merge request pipelines, your `rules` (or [`workflow: rules`](../../../ci/yaml/index.md#workflow)) might look like this example: ```yaml @@ -334,7 +311,7 @@ do this: 1. Define a job in your `.gitlab-ci.yml` file that generates the [Code Quality report - artifact](../../../ci/yaml/README.md#artifactsreportscodequality). + artifact](../../../ci/yaml/index.md#artifactsreportscodequality). 1. Configure your tool to generate the Code Quality report artifact as a JSON file that implements a subset of the [Code Climate spec](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types). @@ -342,13 +319,13 @@ do this: The Code Quality report artifact JSON file must contain an array of objects with the following properties: -| Name | Description | -| ---------------------- | -------------------------------------------------------------------------------------- | -| `description` | A description of the code quality violation. | -| `fingerprint` | A unique fingerprint to identify the code quality violation. For example, an MD5 hash. | -| `severity` | A severity string (can be `info`, `minor`, `major`, `critical`, or `blocker`). | -| `location.path` | The relative path to the file containing the code quality violation. | -| `location.lines.begin` | The line on which the code quality violation occurred. | +| Name | Description | +| ---------------------- | ----------------------------------------------------------------------------------------- | +| `description` | A description of the code quality violation. | +| `fingerprint` | A unique fingerprint to identify the code quality violation. For example, an MD5 hash. | +| `severity` | A severity string (can be `info`, `minor`, `major`, `critical`, or `blocker`). | +| `location.path` | The relative path to the file containing the code quality violation. | +| `location.lines.begin` or `location.positions.begin.line` | The line on which the code quality violation occurred. | Example: @@ -371,6 +348,8 @@ Example: NOTE: Although the Code Climate spec supports more properties, those are ignored by GitLab. +The GitLab parser does not allow a [byte order mark](https://en.wikipedia.org/wiki/Byte_order_mark) +at the beginning of the file. ## Code Quality reports **(PREMIUM)** @@ -389,7 +368,7 @@ After the Code Quality job completes: - The full list of code quality violations generated by a pipeline is shown in the Code Quality tab of the Pipeline Details page. **(PREMIUM)** -### Generating an HTML report +## Generate an HTML report In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/ci-cd/codequality/-/issues/10), it is possible to generate an HTML report file by setting the `REPORT_FORMAT` @@ -535,7 +514,7 @@ This can be due to multiple reasons: - Your pipeline is not set to run the code quality job on your target branch. If there is no report generated from the target branch, your MR branch reports have nothing to compare to. - If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks), nothing is displayed. -- The [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) CI/CD +- The [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) CI/CD setting can cause the Code Quality artifact(s) to expire faster than desired. - The widgets use the pipeline of the latest commit to the target branch. If commits are made to the default branch that do not run the code quality job, this may cause the merge request widget to have no base report for comparison. - If you use the [`REPORT_STDOUT` environment variable](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables), no report file is generated and nothing displays in the merge request. diff --git a/doc/user/project/merge_requests/commits.md b/doc/user/project/merge_requests/commits.md index 1bda12468a3..fb1b7f8b9b6 100644 --- a/doc/user/project/merge_requests/commits.md +++ b/doc/user/project/merge_requests/commits.md @@ -26,3 +26,62 @@ To seamlessly navigate among commits in a merge request: - Using the <kbd>X</kbd> and <kbd>C</kbd> keyboard shortcuts.  + +## View merge request commits in context + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29274) in GitLab 13.12. +> - [Deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - Disabled on GitLab.com. +> - Not recommended for production use. +> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-viewing-merge-request-commits-in-context). **(FREE SELF)** + +WARNING: +This feature is in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +and is [incomplete](https://gitlab.com/groups/gitlab-org/-/epics/1192). +Previously merged commits can be added, but they can't be removed due to +[this bug](https://gitlab.com/gitlab-org/gitlab/-/issues/325538). + +This in-development feature might not be available for your use. There can be +[risks when enabling features still in development](../../feature_flags.md#risks-when-enabling-features-still-in-development). +Refer to this feature's version history for more details. + +When reviewing a merge request, it helps to have more context about the changes +made. That includes unchanged lines in unchanged files, and previous commits +that have already merged that the change is built on. + +To add previously merged commits to a merge request for more context: + +1. Go to your merge request. +1. Select the **Commits** tab. +1. Scroll to the end of the list of commits, and select **Add previously merged commits**: + +  + +1. Select the commits that you want to add. +1. Select **Save changes**. + +To view the changes done on those previously merged commits: + +1. On your merge request, select the **Changes** tab. +1. Scroll to **(file-tree)** **Compare** and select **previously merged commits**: + +  + +### Enable or disable viewing merge request commits in context **(FREE SELF)** + +Viewing merge request commits in context is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:context_commits) +``` + +To disable it: + +```ruby +Feature.disable(:context_commits) +``` diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index 430c6488b26..0d56fbc89b8 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -7,189 +7,155 @@ description: "How to create merge requests in GitLab." disqus_identifier: 'https://docs.gitlab.com/ee/gitlab-basics/add-merge-request.html' --- -# How to create a merge request **(FREE)** +# Creating merge requests **(FREE)** -Before creating a merge request, read through an -[introduction to merge requests](getting_started.md) -to familiarize yourself with the concept, the terminology, -and to learn what you can do with them. +There are many different ways to create a merge request. -Every merge request starts by creating a branch. You can either -do it locally through the [command line](#new-merge-request-from-your-local-environment), via a Git CLI application, -or through the [GitLab UI](#new-merge-request-from-a-new-branch-created-through-the-ui). +## From the merge request list -This document describes the several ways to create a merge request. +You can create a merge request from the list of merge requests. -When you start a new merge request, regardless of the method, -you are taken to the [**New merge request** page](#new-merge-request-page) -to fill it with information about the merge request. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left menu, select **Merge requests**. +1. In the top right, select **New merge request**. +1. Select a source and target branch and then **Compare branches and continue**. +1. Fill out the fields and select **Create merge request**. -If you push a new branch to GitLab, also regardless of the method, -you can click the [**Create merge request**](#create-merge-request-button) -button and start a merge request from there. +## From an issue -## New merge request page +You can [create a merge request from an issue](../repository/web_editor.md#create-a-new-branch-from-an-issue). -On the **New merge request** page, start by filling in the title and description -for the merge request. If commits already exist on the branch, GitLab suggests a -merge request title for you: +## When you add, edit, or upload a file -- **If a multi-line commit message exists**: GitLab adds the first line of the - first multi-line commit message as the title. Any additional lines in that - commit message become the description. -- **If no multi-line commit message exists**: GitLab adds the branch name as the - title, and leaves the description blank. +You can create a merge request when you add, edit, or upload a file to a repository. -The title is the only field that is mandatory in all cases. +1. Add, edit, or upload a file to the repository. +1. In the **Commit message**, enter a reason for the commit. +1. Select the **Target branch** or create a new branch by typing the name (without spaces, capital letters, or special chars). +1. Select the **Start a new merge request with these changes** checkbox or toggle. This checkbox or toggle is visible only + if the target is not the same as the source branch, or if the source branch is protected. +1. Select **Commit changes**. -From there, you can fill it with information (title, description, -assignee(s), milestone, labels, approvers) and click **Create merge request**. +## When you create a branch -From that initial screen, you can also see all the commits, -pipelines, and file changes pushed to your branch before submitting -the merge request. +You can create a merge request when you create a branch. - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left menu, select **Repository > Branches**. +1. Type a branch name and select **New branch**. +1. Above the file list, on the right side, select **Create merge request**. + A merge request is created. The default branch is the target. +1. Fill out the fields and select **Create merge request**. -NOTE: -You can push one or more times to your branch in GitLab before -creating the merge request. +## When you use Git commands locally -## Create merge request button +You can create a merge request by running Git commands on your local machine. -Once you have pushed a new branch to GitLab, visit your repository -in GitLab and to see a call-to-action at the top of your screen -from which you can click the button **Create merge request**. +1. Create a branch: - + ```shell + git checkout -b my-new-branch + ``` -You can also see the **Create merge request** button in the top-right of the: +1. Create, edit, or delete files. The stage and commit them: -- **Project** page. -- **Repository > Files** page. -- **Merge requests** page. + ```shell + git add . + git commit -m "My commit message" + ``` -In this case, GitLab uses the most recent branch you pushed -changes to as the source branch, and the default branch in the current -project as the target. +1. [Push your branch to GitLab](../../../gitlab-basics/start-using-git.md#send-changes-to-gitlabcom): -## New merge request by adding, editing, and uploading a file + ```shell + git push origin my-new-branch + ``` -When you choose to edit, add, or upload a file through the GitLab UI, -at the end of the file you see the option to add the **Commit message**, -to select the **Target branch** of that commit, and the checkbox to -**Start new a merge request with these changes**. + GitLab prompts you with a direct link for creating a merge request: -Similarly, if you change files through the Web IDE, when you navigate to **Commit** on the left-hand sidebar, you see these same options. + ```plaintext + ... + remote: To create a merge request for docs-new-merge-request, visit: + remote: https://gitlab.example.com/my-group/my-project/merge_requests/new?merge_request%5Bsource_branch%5D=my-new-branch + ``` -Once you have added, edited, or uploaded the file: +1. Copy the link and paste it in your browser. -1. Describe your changes in the commit message. -1. Select an existing branch to add your commit into, or, if you'd like to create a new branch, type the new branch name (without spaces, capital letters, or special chars). -1. Keep the checkbox checked to start a new merge request straightaway, or, uncheck it to add more changes to that branch before starting the merge request. -1. Click **Commit changes**. +You can add other [flags to commands when pushing through the command line](../push_options.md) +to reduce the need for editing merge requests manually through the UI. -If you chose to start a merge request, you are taken to the -[**New merge request** page](#new-merge-request-page), from -which you can fill it in with information and submit the merge request. +## When you work in a fork -The merge request targets the default branch of the repository. -If you want to change it, you can do it later by editing the merge request. +You can create a merge request from your fork to contribute back to the main project. -## New merge request from a new branch created through the UI - -To quickly start working on files through the GitLab UI, -navigate to your project's **Repository > Branches** and click -**New branch**. A new branch is created and you can start -editing files. - -Once committed and pushed, you can click on the [**Create merge request**](#create-merge-request-button) -button to open the [**New merge request** page](#new-merge-request-page). -A new merge request is started using the current branch as the source, -and the default branch in the current project as the target. - -## New merge request from your local environment - -Assuming you have your repository cloned into your computer and you'd -like to start working on changes to files, start by creating and -checking out a new branch: - -```shell -git checkout -b my-new-branch -``` - -Work on your file changes, stage, and commit them: +1. On the top bar, select **Menu > Project**. +1. Select your fork of the repository. +1. On the left menu, go to **Merge requests**, and select **New merge request**. +1. In the **Source branch** drop-down list box, select the branch in your forked repository as the source branch. +1. In the **Target branch** drop-down list box, select the branch from the upstream repository as the target branch. + You can set a [default target project](#set-the-default-target-project) to + change the default target branch (which can be useful if you are working in a + forked project). +1. Select **Compare branches and continue**. +1. Select **Submit merge request**. -```shell -git add . -git commit -m "My commit message" -``` +After your work is merged, if you don't intend to +make any other contributions to the upstream project, you can unlink your +fork from its upstream project. Go to **Settings > Advanced Settings** and +[remove the forking relationship](../settings/index.md#removing-a-fork-relationship). -Once you're done, [push your branch to GitLab](../../../gitlab-basics/start-using-git.md#send-changes-to-gitlabcom): +For more information, [see the forking workflow documentation](../repository/forking_workflow.md). -```shell -git push origin my-new-branch -``` +## By sending an email **(FREE SELF)** -In the output, GitLab prompts you with a direct link for creating -a merge request: +> The format of the generated email address changed in GitLab 11.7. + The earlier format is still supported so existing aliases + or contacts still work. -```shell -... -remote: To create a merge request for docs-new-merge-request, visit: -remote: https://gitlab-instance.com/my-group/my-project/merge_requests/new?merge_request%5Bsource_branch%5D=my-new-branch -``` +You can create a merge request by sending an email message to GitLab. +The merge request target branch is the project's default branch. -Copy that link and paste it in your browser, and the [**New merge request page**](#new-merge-request-page) -is displayed. +Prerequisites: -There is also a number of [flags you can add to commands when pushing through the command line](../push_options.md) to reduce the need for editing merge requests manually through the UI. +- A GitLab administrator must configure [incoming email](../../../administration/incoming_email.md). +- A GitLab administrator must configure [Reply by email](../../../administration/reply_by_email.md). -If you didn't push your branch to GitLab through the command line -(for example, you used a Git CLI application to push your changes), -you can create a merge request through the GitLab UI by clicking -the [**Create merge request**](#create-merge-request-button) button. +To create a merge request by sending an email: -## New merge request from an issue +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left menu, select **Merge requests**. +1. In the top right, select **Email a new merge request to this project**. + An email address is displayed. Copy this address. + Ensure you keep this address private. +1. Open an email and compose a message with the following information: -You can also [create a new merge request directly from an issue](../repository/web_editor.md#create-a-new-branch-from-an-issue). + - The **To** line is the email address you copied. + - The subject line is the source branch name. + - The message body is the merge request description. -## New merge request from the merge requests page +1. Send the email message. -You can start creating a new merge request by clicking the -**New merge request** button on the **merge requests** page in a project. -Then choose the source project and branch that contain your changes, -and the target project and branch where you want to merge the changes into. -Click on **Compare branches and continue** to go to the -[**New merge request** page](#new-merge-request-page) and fill in the details. +A merge request is created. -## New merge request from a fork +### Add attachments when creating a merge request by email -After forking a project and applying your local changes, complete the following steps to -create a merge request from your fork to contribute back to the main project: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22723) in GitLab 11.5. -1. On the top bar, select **Menu > Project**. -1. Select **Your Projects**, then select your fork of the repository. -1. In the left menu, go to **Merge requests**, and click **New merge request**. -1. In the **Source branch** drop-down list box, select your branch in your forked repository as the source branch. -1. In the **Target branch** drop-down list box, select the branch from the upstream repository as the target branch. - You can set a [default target project](#set-the-default-target-project) to - change the default target branch (which can be useful when working with a - forked project). -1. After entering the credentials, click **Compare branches and continue** to compare your local changes to the upstream repository. -1. Assign a user to review your changes, and click **Submit merge request**. +You can add commits to a merge request by adding +patches as attachments to the email. All attachments with a filename +ending in `.patch` are considered patches and are processed +ordered by name. -When the changes are merged, your changes are added to the upstream repository and -the branch as per specification. After your work is merged, if you don't intend to -make any other contributions to the upstream project, you can unlink your -fork from its upstream project in the **Settings > Advanced Settings** section by -[removing the forking relationship](../settings/index.md#removing-a-fork-relationship). +The combined size of the patches can be 2 MB. -For further details, [see the forking workflow documentation](../repository/forking_workflow.md). +If the source branch from the subject does not exist, it is +created from the repository's HEAD or the specified target branch. +You can specify the target branch by using the +[`/target_branch` quick action](../quick_actions.md). If the source +branch already exists, the patches are applied on top of it. ## Set the default target project -Merge requests have a source and a target project which are the same, unless +Merge requests have a source and a target project that are the same, unless forking is involved. Creating a fork of the project can cause either of these scenarios when you create a new merge request: @@ -197,57 +163,11 @@ scenarios when you create a new merge request: option). - You target your own fork. -If you want to have merge requests from a fork by default target your own fork -(instead of the upstream project), you can change the default by: +To have merge requests from a fork by default target your own fork +(instead of the upstream project), you can change the default. -1. In your project, go to **Settings > General > Merge requests**. +1. On the top bar, select **Menu > Project**. +1. On the left menu, select **Settings > General > Merge requests**. 1. In the **Target project** section, select the option you want to use for your default target project. 1. Select **Save changes**. - -## New merge request by email **(FREE SELF)** - -_This feature needs [incoming email](../../../administration/incoming_email.md) -to be configured by a GitLab administrator to be available._ It isn't -available in GitLab.com. - -You can create a new merge request by sending an email to a user-specific email -address. The address can be obtained on the merge requests page by clicking on -a **Email a new merge request to this project** button. The subject is -used as the source branch name for the new merge request and the target branch -is the default branch for the project. The message body (if not empty) -is used as the merge request description. You need -["Reply by email"](../../../administration/reply_by_email.md) enabled to use -this feature. If it's not enabled to your instance, you may ask your GitLab -administrator to do so. - -This is a private email address, generated just for you. **Keep it to yourself** -as anyone who has it can create issues or merge requests as if they were you. -You can add this address to your contact list for easy access. - - - -_In GitLab 11.7, we updated the format of the generated email address. -However the older format is still supported, allowing existing aliases -or contacts to continue working._ - -### Adding patches when creating a merge request via e-mail - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22723) in GitLab 11.5. - -You can add commits to the merge request being created by adding -patches as attachments to the email. All attachments with a filename -ending in `.patch` are considered patches and they are processed -ordered by name. - -The combined size of the patches can be 2MB. - -If the source branch from the subject does not exist, it is -created from the repository's HEAD or the specified target branch to -apply the patches. The target branch can be specified using the -[`/target_branch` quick action](../quick_actions.md). If the source -branch already exists, the patches are applied on top of it. - -## Reviewing and managing merge requests - -Once you have submitted a merge request, it can be [reviewed and managed](reviews/index.md) through GitLab. diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 57850ad7304..a91679ffd63 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -79,9 +79,9 @@ draft merge requests: ## Pipelines for drafts -When the [pipelines for merged results](../../../ci/merge_request_pipelines/pipelines_for_merged_results/index.md) +When the [pipelines for merged results](../../../ci/pipelines/pipelines_for_merged_results.md) feature is enabled, draft merge requests run -[merge request pipelines](../../../ci/merge_request_pipelines/index.md) only. +[merge request pipelines](../../../ci/pipelines/merge_request_pipelines.md) only. To run pipelines for merged results, you must [mark the merge request as ready](#mark-merge-requests-as-ready). diff --git a/doc/user/project/merge_requests/fail_fast_testing.md b/doc/user/project/merge_requests/fail_fast_testing.md index 68a63aebb90..15ab2d9c8e2 100644 --- a/doc/user/project/merge_requests/fail_fast_testing.md +++ b/doc/user/project/merge_requests/fail_fast_testing.md @@ -19,7 +19,7 @@ that it believes to be relevant to the input files. `tff` is designed for Ruby on Rails projects, so the `Verify/FailFast` template is configured to run when changes to Ruby files are detected. By default, it runs in -the [`.pre` stage](../../../ci/yaml/README.md#pre-and-post) of a GitLab CI/CD pipeline, +the [`.pre` stage](../../../ci/yaml/index.md#pre-and-post) of a GitLab CI/CD pipeline, before all other stages. ## Example use case @@ -42,8 +42,8 @@ This template requires: - A project built in Rails that uses RSpec for testing. - CI/CD configured to: - Use a Docker image with Ruby available. - - Use [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md#configuring-pipelines-for-merge-requests) -- [Pipelines for Merged Results](../../../ci/merge_request_pipelines/pipelines_for_merged_results/index.md#enable-pipelines-for-merged-results) + - Use [Pipelines for merge requests](../../../ci/pipelines/merge_request_pipelines.md#configure-pipelines-for-merge-requests) +- [Pipelines for Merged Results](../../../ci/pipelines/pipelines_for_merged_results.md#enable-pipelines-for-merged-results) enabled in the project settings. - A Docker image with Ruby available. The template uses `image: ruby:2.6` by default, but you [can override](../../../ci/yaml/includes.md#overriding-external-template-values) this. @@ -62,7 +62,7 @@ rspec-complete: - bundle exec rspec ``` -To run the most relevant specs first instead of the whole suite, [`include`](../../../ci/yaml/README.md#include) +To run the most relevant specs first instead of the whole suite, [`include`](../../../ci/yaml/index.md#include) the template by adding the following to your CI/CD configuration: ```yaml diff --git a/doc/user/project/merge_requests/img/add_previously_merged_commits_button_v14_1.png b/doc/user/project/merge_requests/img/add_previously_merged_commits_button_v14_1.png Binary files differnew file mode 100644 index 00000000000..e60e869f854 --- /dev/null +++ b/doc/user/project/merge_requests/img/add_previously_merged_commits_button_v14_1.png diff --git a/doc/user/project/merge_requests/img/code_quality_mr_diff_report_v14.png b/doc/user/project/merge_requests/img/code_quality_mr_diff_report_v14.png Binary files differindex a942420d65e..da7d4115bd9 100644 --- a/doc/user/project/merge_requests/img/code_quality_mr_diff_report_v14.png +++ b/doc/user/project/merge_requests/img/code_quality_mr_diff_report_v14.png diff --git a/doc/user/project/merge_requests/img/create_from_email.png b/doc/user/project/merge_requests/img/create_from_email.png Binary files differdeleted file mode 100644 index 14eef473e27..00000000000 --- a/doc/user/project/merge_requests/img/create_from_email.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/create_merge_request_button_v12_6.png b/doc/user/project/merge_requests/img/create_merge_request_button_v12_6.png Binary files differdeleted file mode 100644 index bcbee10e1b7..00000000000 --- a/doc/user/project/merge_requests/img/create_merge_request_button_v12_6.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/new_merge_request_page_v12_6.png b/doc/user/project/merge_requests/img/new_merge_request_page_v12_6.png Binary files differdeleted file mode 100644 index c0f2ba261cb..00000000000 --- a/doc/user/project/merge_requests/img/new_merge_request_page_v12_6.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/previously_merged_commits_v14_1.png b/doc/user/project/merge_requests/img/previously_merged_commits_v14_1.png Binary files differnew file mode 100644 index 00000000000..4f49fad10ad --- /dev/null +++ b/doc/user/project/merge_requests/img/previously_merged_commits_v14_1.png diff --git a/doc/user/project/merge_requests/img/status_checks_widget_passed_v14_0.png b/doc/user/project/merge_requests/img/status_checks_widget_passed_v14_0.png Binary files differnew file mode 100644 index 00000000000..de61ca8b553 --- /dev/null +++ b/doc/user/project/merge_requests/img/status_checks_widget_passed_v14_0.png diff --git a/doc/user/project/merge_requests/img/status_checks_widget_pending_v14_0.png b/doc/user/project/merge_requests/img/status_checks_widget_pending_v14_0.png Binary files differnew file mode 100644 index 00000000000..c4e606bd2f4 --- /dev/null +++ b/doc/user/project/merge_requests/img/status_checks_widget_pending_v14_0.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index b5c51c42ae9..fa90cf524ec 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -27,13 +27,13 @@ important parts of the merge request:  - **Overview**: Contains the description, notifications from pipelines, and a - discussion area for [comment threads](../../discussions/index.md#resolvable-comments-and-threads) + discussion area for [comment threads](../../discussions/index.md#resolve-a-thread)) and [code suggestions](reviews/suggestions.md). The right sidebar provides fields to add assignees, reviewers, labels, and a milestone to your work, and the [merge request widgets area](widgets.md) reports results from pipelines and tests. - **Commits**: Contains a list of commits added to this merge request. For more information, read [Commits tab in merge requests](commits.md). -- **Pipelines**: If configured, contains a list of recent [GitLab CI/CD](../../../ci/README.md) +- **Pipelines**: If configured, contains a list of recent [GitLab CI/CD](../../../ci/index.md) pipelines and their status. - **Changes**: Contains the diffs of files changed by this merge request. You can [configure the display](changes.md). @@ -119,7 +119,7 @@ For a software developer working in a team: 1. Pushes a commit with their final review. 1. [Approves the merge request](approvals/index.md). 1. Sets it to [merge when pipeline succeeds](merge_when_pipeline_succeeds.md). -1. Your changes get deployed to production with [manual actions](../../../ci/yaml/README.md#whenmanual) for GitLab CI/CD. +1. Your changes get deployed to production with [manual actions](../../../ci/yaml/index.md#whenmanual) for GitLab CI/CD. 1. Your implementations were successfully shipped to your customer. For a web developer writing a webpage for your company's website: diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index d1b697add08..7ea785c00ea 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -10,7 +10,7 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. With Load Performance Testing, you can test the impact of any pending code changes -to your application's backend in [GitLab CI/CD](../../../ci/README.md). +to your application's backend in [GitLab CI/CD](../../../ci/index.md). GitLab uses [k6](https://k6.io/), a free and open source tool, for measuring the system performance of applications under @@ -28,7 +28,7 @@ GET calls to a popular API endpoint in your application to see how it performs. ## How Load Performance Testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Load Performance report artifact](../../../ci/yaml/README.md#artifactsreportsload_performance). +[Load Performance report artifact](../../../ci/yaml/index.md#artifactsreportsload_performance). GitLab checks this report, compares key load performance metrics between the source and target branches, and then shows the information in a merge request widget: @@ -93,7 +93,7 @@ template that is included with GitLab. NOTE: For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) -for spec details. The [default shared GitLab.com runners](../../../ci/runners/README.md#linux-shared-runners) +for spec details. The [default shared GitLab.com runners](../../../ci/runners/build_cloud/linux_build_cloud.md) likely have insufficient specs to handle most large k6 tests. This template runs the @@ -140,7 +140,7 @@ For example, you can override the duration of the test with a CLI option: GitLab only displays the key performance metrics in the MR widget if k6's results are saved via [summary export](https://k6.io/docs/results-visualization/json#summary-export) -as a [Load Performance report artifact](../../../ci/yaml/README.md#artifactsreportsload_performance). +as a [Load Performance report artifact](../../../ci/yaml/index.md#artifactsreportsload_performance). The latest Load Performance artifact available is always used, using the summary values from the test. diff --git a/doc/user/project/merge_requests/merge_request_dependencies.md b/doc/user/project/merge_requests/merge_request_dependencies.md index 21282a55ff2..aace1f58c62 100644 --- a/doc/user/project/merge_requests/merge_request_dependencies.md +++ b/doc/user/project/merge_requests/merge_request_dependencies.md @@ -27,7 +27,7 @@ the other way around. imports the library. - Prevent a documentation-only merge request from being merged before the merge request implementing the feature to be documented. -- Require an merge request updating a permissions matrix to be merged before merging an +- Require a merge request updating a permissions matrix to be merged before merging a merge request from someone who hasn't yet been granted permissions. It is common for a single logical change to span several merge requests, spread diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 6c1e33a9ace..d7c9c0f73ee 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -21,7 +21,7 @@ request is updated to show the impending merge. If you can't wait for the pipeline to succeed, you can choose **Merge immediately** in the dropdown menu on the right of the main button. -The author of the merge request and project members with developer permissions can +The author of the merge request and project members with the Developer role can cancel the automatic merge at any time before the pipeline finishes.  @@ -67,8 +67,8 @@ You should be careful to configure CI/CD so that pipelines run for every merge r ### Limitations When this setting is enabled, a merge request is prevented from being merged if there -is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#only--except) -or [`rules`](../../../ci/yaml/README.md#rules) are used and they don't generate any pipelines. +is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/index.md#only--except) +or [`rules`](../../../ci/yaml/index.md#rules) are used and they don't generate any pipelines. You should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) and that it's successful. @@ -101,7 +101,7 @@ for details on avoiding two pipelines for a single merge request. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211482) in GitLab 13.1. -When the **Pipelines must succeed** checkbox is checked, [skipped pipelines](../../../ci/yaml/README.md#skip-pipeline) prevent +When the **Pipelines must succeed** checkbox is checked, [skipped pipelines](../../../ci/yaml/index.md#skip-pipeline) prevent merge requests from being merged. To change this behavior: 1. Navigate to your project's **Settings > General** page. diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 317202e9303..16267e13fd5 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -100,7 +100,7 @@ When you submit your review, GitLab: ### Resolving/Unresolving threads -Review comments can also resolve or unresolve [resolvable threads](../../../discussions/index.md#resolvable-comments-and-threads). +Review comments can also resolve or unresolve [resolvable threads](../../../discussions/index.md#resolve-a-thread)). When replying to a comment, a checkbox is displayed to resolve or unresolve the thread after publication. diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md index 9409cc569a6..8ee068531c8 100644 --- a/doc/user/project/merge_requests/reviews/suggestions.md +++ b/doc/user/project/merge_requests/reviews/suggestions.md @@ -14,7 +14,7 @@ type: index, reference As a reviewer, you're able to suggest code changes with a Markdown syntax in merge request diff threads. Then, the merge request author (or other users with appropriate [permission](../../../permissions.md)) is able to apply these suggestions with a click, -which generates a commit in the merge request authored by the user that applied them. +which generates a commit in the merge request authored by the user that suggested the changes. 1. Choose a line of code to be changed, add a new comment, then select the **Insert suggestion** icon in the toolbar: @@ -42,7 +42,7 @@ which generates a commit in the merge request authored by the user that applied After the author applies a suggestion, it's marked with the **Applied** label, the thread is automatically resolved, and GitLab creates a new commit and pushes the suggested change directly into the codebase in the merge request's -branch. [Developer permission](../../../permissions.md) is required to do so. +branch. The [Developer role](../../../permissions.md) is required to do so. ## Multi-line suggestions diff --git a/doc/user/project/merge_requests/status_checks.md b/doc/user/project/merge_requests/status_checks.md index 775820870f3..70e2d718406 100644 --- a/doc/user/project/merge_requests/status_checks.md +++ b/doc/user/project/merge_requests/status_checks.md @@ -8,11 +8,8 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/merge_requests/statu # External Status Checks **(ULTIMATE)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3869) in GitLab 14.0. -> - It's [deployed behind a feature flag](../../feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-status-checks). **(ULTIMATE SELF)** +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3869) in GitLab 14.0, disabled behind the `:ff_external_status_checks` feature flag. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/320783) in GitLab 14.1. WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -113,6 +110,31 @@ To complete the deletion of the status check you must select the **Remove status check** button. This **permanently** deletes the status check and it **will not** be recoverable. +## Status checks widget + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327634) in GitLab 14.1. + +The status checks widget displays in merge requests and shows the status of external +status checks: + + + +An organization might have a policy that does not allow merging merge requests if +external status checks do not pass. However, the details in the widget are for informational +purposes only. GitLab does not prevent merging of merge requests that fail status checks. + +While GitLab waits for a response from the external status check, the widget shows +the status checks as `pending`: + + + +After GitLab [receives a response](../../../api/status_checks.md#set-approval-status-of-an-external-status-check) +from the external status check, the widget updates accordingly. + +NOTE: +GitLab cannot guarantee that the external status checks are properly processed by +the related external service. + ## Troubleshooting ### Duplicate value errors @@ -149,30 +171,18 @@ An unexpected response was received from the branches retrieval API. As suggested, you should close the form and reopen again or refresh the page. This error should be temporary, although if it persists please check the [GitLab status page](https://status.gitlab.com/) to see if there is a wider outage. -## Enable or disable status checks **(ULTIMATE SELF)** - -Status checks are under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it. +### Failed to load status checks -To enable it: - -```ruby -# For the instance -Feature.enable(:ff_compliance_approval_gates) -# For a single project -Feature.enable(:ff_compliance_approval_gates, Project.find(<project id>)) +```plaintext +Failed to load status checks ``` -To disable it: +An unexpected response was received from the external status checks API. +You should: -```ruby -# For the instance -Feature.disable(:ff_compliance_approval_gates) -# For a single project -Feature.disable(:ff_compliance_approval_gates, Project.find(<project id>) -``` +- Refresh the page in case this error is temporary. +- Check the [GitLab status page](https://status.gitlab.com/) if the problem persists, + to see if there is a wider outage. ## Related links diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index e044d50d246..ce8bfa2d054 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -10,7 +10,7 @@ type: reference, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249811) in GitLab 13.5. -With the help of [GitLab CI/CD](../../../ci/README.md), you can collect the test +With the help of [GitLab CI/CD](../../../ci/index.md), you can collect the test coverage information of your favorite testing or coverage-analysis tool, and visualize this information inside the file diff view of your merge requests (MRs). This will allow you to see which lines are covered by tests, and which lines still require coverage, before the @@ -21,14 +21,14 @@ MR is merged. ## How test coverage visualization works Collecting the coverage information is done via GitLab CI/CD's -[artifacts reports feature](../../../ci/yaml/README.md#artifactsreports). +[artifacts reports feature](../../../ci/yaml/index.md#artifactsreports). You can specify one or more coverage reports to collect, including wildcard paths. GitLab then takes the coverage information in all the files and combines it together. For the coverage analysis to work, you have to provide a properly formatted [Cobertura XML](https://cobertura.github.io/cobertura/) report to -[`artifacts:reports:cobertura`](../../../ci/yaml/README.md#artifactsreportscobertura). +[`artifacts:reports:cobertura`](../../../ci/yaml/index.md#artifactsreportscobertura). This format was originally developed for Java, but most coverage analysis frameworks for other languages have plugins to add support for it, like: @@ -129,7 +129,7 @@ The `source` is ignored if the path does not follow this pattern. The parser ass ### JavaScript example -The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example uses [Mocha](https://mochajs.org/) +The following [`gitlab-ci.yml`](../../../ci/yaml/index.md) example uses [Mocha](https://mochajs.org/) JavaScript testing and [nyc](https://github.com/istanbuljs/nyc) coverage-tooling to generate the coverage artifact: @@ -147,7 +147,7 @@ test: #### Maven example -The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for Java or Kotlin uses [Maven](https://maven.apache.org/) +The following [`gitlab-ci.yml`](../../../ci/yaml/index.md) example for Java or Kotlin uses [Maven](https://maven.apache.org/) to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to generate the coverage artifact. You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. @@ -185,7 +185,7 @@ coverage-jdk11: #### Gradle example -The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for Java or Kotlin uses [Gradle](https://gradle.org/) +The following [`gitlab-ci.yml`](../../../ci/yaml/index.md) example for Java or Kotlin uses [Gradle](https://gradle.org/) to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to generate the coverage artifact. You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. @@ -223,7 +223,7 @@ coverage-jdk11: ### Python example -The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for Python uses [pytest-cov](https://pytest-cov.readthedocs.io/) to collect test coverage data and [coverage.py](https://coverage.readthedocs.io/) to convert the report to use full relative paths. +The following [`gitlab-ci.yml`](../../../ci/yaml/index.md) example for Python uses [pytest-cov](https://pytest-cov.readthedocs.io/) to collect test coverage data and [coverage.py](https://coverage.readthedocs.io/) to convert the report to use full relative paths. The information isn't displayed without the conversion. This example assumes that the code for your package is in `src/` and your tests are in `tests.py`: @@ -243,7 +243,7 @@ run tests: ### C/C++ example -The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for C/C++ with +The following [`gitlab-ci.yml`](../../../ci/yaml/index.md) example for C/C++ with `gcc` or `g++` as the compiler uses [`gcovr`](https://gcovr.com/en/stable/) to generate the coverage output file in Cobertura XML format. diff --git a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md index 55e122dec76..0a9a2a37bfe 100644 --- a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md +++ b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md @@ -17,13 +17,13 @@ or link to useful information directly from merge requests: | [Browser Performance Testing](browser_performance_testing.md) **(PREMIUM)** | Quickly determine the browser performance impact of pending code changes. | | [Load Performance Testing](load_performance_testing.md) **(PREMIUM)** | Quickly determine the server performance impact of pending code changes. | | [Code Quality](code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | -| [Display arbitrary job artifacts](../../../ci/yaml/README.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../../../ci/pipelines/job_artifacts.md) in merge requests. | -| [GitLab CI/CD](../../../ci/README.md) | Build, test, and deploy your code in a per-branch basis with built-in CI/CD. | +| [Display arbitrary job artifacts](../../../ci/yaml/index.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../../../ci/pipelines/job_artifacts.md) in merge requests. | +| [GitLab CI/CD](../../../ci/index.md) | Build, test, and deploy your code in a per-branch basis with built-in CI/CD. | | [Unit test reports](../../../ci/unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it's easier and faster to identify the failure without having to check the entire job log. | | [License Compliance](../../compliance/license_compliance/index.md) **(ULTIMATE)** | Manage the licenses of your dependencies. | | [Metrics Reports](../../../ci/metrics_reports.md) **(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. | -| [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. | -| [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. | +| [Multi-Project pipelines](../../../ci/pipelines/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. | +| [Pipelines for merge requests](../../../ci/pipelines/merge_request_pipelines.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. | | [Pipeline Graphs](../../../ci/pipelines/index.md#visualize-pipelines) | View the status of pipelines within the merge request, including the deployment process. | | [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, within the file diff. | diff --git a/doc/user/project/merge_requests/widgets.md b/doc/user/project/merge_requests/widgets.md index 92b2a8f24ef..b0464f3f972 100644 --- a/doc/user/project/merge_requests/widgets.md +++ b/doc/user/project/merge_requests/widgets.md @@ -14,7 +14,7 @@ and the services you configure for your project. ## Pipeline information -If you've set up [GitLab CI/CD](../../../ci/README.md) in your project, +If you've set up [GitLab CI/CD](../../../ci/index.md) in your project, a [merge request](index.md) displays pipeline information in the widgets area of the **Overview** tab: @@ -62,3 +62,9 @@ merge request widget takes you directly to the pages changed, making it easier a faster to preview proposed modifications. [Read more about Review Apps](../../../ci/review_apps/index.md). + +## External status checks **(ULTIMATE)** + +If you have configured [external status checks](status_checks.md) you can +see the status of these checks in merge requests +[in a specific widget](status_checks.md#status-checks-widget). diff --git a/doc/user/project/merge_requests/work_in_progress_merge_requests.md b/doc/user/project/merge_requests/work_in_progress_merge_requests.md deleted file mode 100644 index 8b663b8edf8..00000000000 --- a/doc/user/project/merge_requests/work_in_progress_merge_requests.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'drafts.md' -remove_date: '2021-05-19' ---- - -This document was moved to [another location](drafts.md). - -<!-- This redirect file can be deleted after <2021-05-19>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md deleted file mode 100644 index 55fde63dd47..00000000000 --- a/doc/user/project/new_ci_build_permissions_model.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../ci/README.md' -remove_date: '2021-06-01' ---- - -This document is deprecated. See the latest [GitLab CI/CD documentation](../../ci/README.md). - -<!-- This redirect file can be deleted after <2021-06-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index 9f80e2e7613..eb5f3a1bbf0 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -4,40 +4,54 @@ group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Create a GitLab Pages website from scratch **(FREE)** +# Tutorial: Create a GitLab Pages website from scratch **(FREE)** -This tutorial shows you how to create a Pages site from scratch. You start with -a blank project and create your own CI file, which gives instruction to -a [runner](https://docs.gitlab.com/runner/). When your CI/CD +This tutorial shows you how to create a Pages site from scratch using +the [Jekyll](https://jekyllrb.com/) Static Site Generator (SSG). You start with +a blank project and create your own CI/CD configuration file, which gives +instructions to a [runner](https://docs.gitlab.com/runner/). When your CI/CD [pipeline](../../../../ci/pipelines/index.md) runs, the Pages site is created. -This example uses the [Jekyll](https://jekyllrb.com/) Static Site Generator (SSG). -Other SSGs follow similar steps. You do not need to be familiar with Jekyll or SSGs +This example uses Jekyll, but other SSGs follow similar steps. +You do not need to be familiar with Jekyll or SSGs to complete this tutorial. +To create a GitLab Pages website: + +- [Step 1: Create the project files](#create-the-project-files) +- [Step 2: Choose a Docker image](#choose-a-docker-image) +- [Step 3: Install Jekyll](#install-jekyll) +- [Step 4: Specify the `public` directory for output](#specify-the-public-directory-for-output) +- [Step 5: Specify the `public` directory for artifacts](#specify-the-public-directory-for-artifacts) +- [Step 6: Deploy and view your website](#deploy-and-view-your-website) + ## Prerequisites -To follow along with this example, start with a blank project in GitLab. -Create three files in the root (top-level) directory. +You must have a [blank project](../../working_with_projects.md#blank-projects) in GitLab. -- `.gitlab-ci.yml` A YAML file that contains the commands you want to run. - For now, leave the file's contents blank. +## Create the project files -- `index.html` An HTML file you can populate with whatever HTML content you'd like, - for example: +Create three files in the root (top-level) directory: - ```html - <html> - <head> - <title>Home</title> - </head> - <body> - <h1>Hello World!</h1> - </body> - </html> - ``` +- `.gitlab-ci.yml`: A YAML file that contains the commands you want to run. + For now, leave the file's contents blank. + +- `index.html`: An HTML file you can populate with whatever HTML content + you'd like, for example: + + ```html + <html> + <head> + <title>Home</title> + </head> + <body> + <h1>Hello World!</h1> + </body> + </html> + ``` + +- [`Gemfile`](https://bundler.io/gemfile.html): A file that describes dependencies for Ruby programs. -- [`Gemfile`](https://bundler.io/gemfile.html) A file that describes dependencies for Ruby programs. Populate it with this content: ```ruby @@ -53,7 +67,7 @@ to run scripts and deploy the site. This specific Ruby image is maintained on [DockerHub](https://hub.docker.com/_/ruby). -Edit your `.gitlab-ci.yml` and add this text as the first line. +Edit your `.gitlab-ci.yml` file and add this text as the first line: ```yaml image: ruby:2.7 @@ -65,13 +79,15 @@ image that contains NodeJS as part of its file system. For example, for a ## Install Jekyll -To run [Jekyll](https://jekyllrb.com/) locally, you would open your terminal and: +To run [Jekyll](https://jekyllrb.com/) locally, you must install it: -- Install [Bundler](https://bundler.io/) by running `gem install bundler`. -- Create `Gemfile.lock` by running `bundle install`. -- Install Jekyll by running `bundle exec jekyll build`. +1. Open your terminal. +1. Install [Bundler](https://bundler.io/) by running `gem install bundler`. +1. Create `Gemfile.lock` by running `bundle install`. +1. Install Jekyll by running `bundle exec jekyll build`. -In the `.gitlab-ci.yml` file, this is written as: +To run Jekyll in your project, edit the `.gitlab-ci.yml` file +and add the installation commands: ```yaml script: @@ -109,7 +125,8 @@ pages: Jekyll needs to know where to generate its output. GitLab Pages only considers files in a directory called `public`. -Jekyll uses destination (`-d`) to specify an output directory for the built website: +Jekyll uses a destination flag (`-d`) to specify an output directory for the built website. +Add the destination to your `.gitlab-ci.yml` file: ```yaml pages: @@ -136,7 +153,7 @@ pages: - public ``` -Paste this into `.gitlab-ci.yml` file, so it now looks like this: +Your `.gitlab-ci.yml` file should now look like this: ```yaml image: ruby:2.7 @@ -151,23 +168,29 @@ pages: - public ``` -Now save and commit the `.gitlab-ci.yml` file. You can watch the pipeline run -by going to **CI/CD > Pipelines**. +## Deploy and view your website -When it succeeds, go to **Settings > Pages** to view the URL where your site -is now available. +After you have completed the preceding steps, +deploy your website: + +1. Save and commit the `.gitlab-ci.yml` file. +1. Go to **CI/CD > Pipelines** to watch the pipeline. +1. When the pipeline succeeds, go to **Settings > Pages** + to view the URL where your site is now available. + +When this `pages` job completes successfully, a special `pages:deploy` job +appears in the pipeline view. It prepares the content of the website for the +GitLab Pages daemon. GitLab runs it in the background and doesn't use a runner. + +## Other options for your CI/CD file If you want to do more advanced tasks, you can update your `.gitlab-ci.yml` file -with [any of the available settings](../../../../ci/yaml/README.md). You can validate +with [any of the available settings](../../../../ci/yaml/index.md). You can validate your `.gitlab-ci.yml` file with the [CI Lint](../../../../ci/lint.md) tool that's included with GitLab. -After successful execution of this `pages` job, a special `pages:deploy` job appears in the -pipeline view. It prepares the content of the website for GitLab Pages daemon. GitLab executes it in -the background and doesn't use runner. - The following topics show other examples of other options you can add to your CI/CD file. -## Deploy specific branches to a Pages site +### Deploy specific branches to a Pages site You may want to deploy to a Pages site only from specific branches. @@ -191,7 +214,8 @@ pages: - public ``` -Then configure the pipeline to run the job for the `master` branch only. +Then configure the pipeline to run the job for the +[default branch](../../repository/branches/default.md) (here, `main`) only. ```yaml image: ruby:2.7 @@ -209,17 +233,17 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "master"' + - if: '$CI_COMMIT_BRANCH == "main"' ``` -## Specify a stage to deploy +### Specify a stage to deploy There are three default stages for GitLab CI/CD: build, test, and deploy. If you want to test your script and check the built site before deploying to production, you can run the test exactly as it runs when you -push to `master`. +push to your [default branch](../../repository/branches/default.md) (here, `main`). To specify a stage for your job to run in, add a `stage` line to your CI file: @@ -241,11 +265,11 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "master"' + - if: '$CI_COMMIT_BRANCH == "main"' ``` Now add another job to the CI file, telling it to -test every push to every branch **except** the `master` branch: +test every push to every branch **except** the `main` branch: ```yaml image: ruby:2.7 @@ -264,7 +288,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "master"' + - if: '$CI_COMMIT_BRANCH == "main"' test: stage: test @@ -276,19 +300,19 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "master"' + - if: '$CI_COMMIT_BRANCH != "main"' ``` When the `test` job runs in the `test` stage, Jekyll builds the site in a directory called `test`. The job affects -all branches except `master`. +all branches except `main`. When you apply stages to different jobs, every job in the same stage builds in parallel. If your web application needs more than one test before being deployed, you can run all your tests at the same time. -## Remove duplicate commands +### Remove duplicate commands To avoid duplicating the same scripts in every job, you can add them to a `before_script` section. @@ -317,7 +341,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "master"' + - if: '$CI_COMMIT_BRANCH == "main"' test: stage: test @@ -327,10 +351,10 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "master"' + - if: '$CI_COMMIT_BRANCH != "main"' ``` -## Build faster with cached dependencies +### Build faster with cached dependencies To build faster, you can cache the installation files for your project's dependencies by using the `cache` parameter. @@ -361,7 +385,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "master"' + - if: '$CI_COMMIT_BRANCH == "main"' test: stage: test @@ -371,7 +395,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "master"' + - if: '$CI_COMMIT_BRANCH != "main"' ``` In this case, you need to exclude the `/vendor` @@ -386,10 +410,11 @@ exclude: - vendor ``` -Now GitLab CI/CD not only builds the website, -but also pushes with **continuous tests** to feature-branches, -**caches** dependencies installed with Bundler, and -**continuously deploys** every push to the `master` branch. +Now GitLab CI/CD not only builds the website, but also: + +- Pushes with **continuous tests** to feature branches. +- **Caches** dependencies installed with Bundler. +- **Continuously deploys** every push to the `main` branch. ## Related topics diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 1f5e1a6ab45..5a688fbb485 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -78,7 +78,7 @@ GitLab always deploys your website from a very specific folder called `public` i repository. When you create a new project in GitLab, a [repository](../repository/index.md) becomes available automatically. -To deploy your site, GitLab uses its built-in tool called [GitLab CI/CD](../../../ci/README.md) +To deploy your site, GitLab uses its built-in tool called [GitLab CI/CD](../../../ci/index.md) to build your site and publish it to the GitLab Pages server. The sequence of scripts that GitLab CI/CD runs to accomplish this task is created from a file named `.gitlab-ci.yml`, which you can [create and modify](getting_started/pages_from_scratch.md). diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 4d6a8653657..94656c91e98 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -22,7 +22,7 @@ In brief, this is what you need to upload your website in GitLab Pages: 1. Domain of the instance: domain name that is used for GitLab Pages (ask your administrator). -1. GitLab CI/CD: a `.gitlab-ci.yml` file with a specific job named [`pages`](../../../ci/yaml/README.md#pages) in the root directory of your repository. +1. GitLab CI/CD: a `.gitlab-ci.yml` file with a specific job named [`pages`](../../../ci/yaml/index.md#pages) in the root directory of your repository. 1. A directory called `public` in your site's repository containing the content to be published. 1. GitLab Runner enabled for the project. @@ -129,7 +129,7 @@ See this document for a [step-by-step guide](getting_started/pages_from_scratch. Remember that GitLab Pages are by default branch/tag agnostic and their deployment relies solely on what you specify in `.gitlab-ci.yml`. You can limit -the `pages` job with the [`only` parameter](../../../ci/yaml/README.md#only--except), +the `pages` job with the [`only` parameter](../../../ci/yaml/index.md#only--except), whenever a new commit is pushed to a branch used specifically for your pages. diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md index ce49699785e..978e35b3a9f 100644 --- a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -60,7 +60,7 @@ might be slightly different. Follow the sudo certbot certonly -a manual -d example.com --email your@email.com ``` - Alternatively, you can register without adding an e-mail account, + Alternatively, you can register without adding an email account, but you aren't notified about the certificate expiration's date: ```shell diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 4b77236f808..4ff651891b2 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -13,24 +13,24 @@ further restrictions on certain branches, they can be protected. The default branch for your repository is protected by default. -## Who can access a protected branch +## Who can modify a protected branch -When a branch is protected, the default behavior enforces -these restrictions on the branch. +When a branch is protected, the default behavior enforces these restrictions on the branch. -| Action | Who can do it | -|--------------------------|---------------| -| Protect a branch | Maintainers only. | -| Push to the branch | GitLab administrators and anyone with **Allowed** permission. (*) | -| Force push to the branch | No one. | -| Delete the branch | No one. | +| Action | Who can do it | +|:-------------------------|:------------------------------------------------------------------| +| Protect a branch | Maintainers only. | +| Push to the branch | GitLab administrators and anyone with **Allowed** permission. (1) | +| Force push to the branch | No one. | +| Delete the branch | No one. | -(*) Users with developer permissions can create a project in a group, -but might not be allowed to initially push to the [default branch](repository/branches/default.md). +1. Users with the Developer role can create a project in a group, but might not be allowed to + initially push to the [default branch](repository/branches/default.md). -### Set the branch protection default level +### Set the default branch protection level -The default branch protection level is set in the [Admin Area](../admin_area/settings/visibility_and_access_controls.md#default-branch-protection). +Administrators can set a default branch protection level in the +[Admin Area](../admin_area/settings/visibility_and_access_controls.md#default-branch-protection). ## Configure a protected branch @@ -43,140 +43,108 @@ To protect a branch: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. +1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. 1. Select **Protect**. -The protected branch displays in the **Protected branches** list. +The protected branch displays in the list of protected branches. -## Using the Allowed to merge and Allowed to push settings +## Configure multiple protected branches by using a wildcard -In GitLab 8.11 and later, we added another layer of branch protection which provides -more granular management of protected branches. The **Developers can push** -option was replaced by **Allowed to push**. You can set this value to allow -or prohibit Maintainers and/or Developers to push to a protected branch. - -Using the **Allowed to push** and **Allowed to merge** settings, you can control -the actions that different roles can perform with the protected branch. -For example, you could set **Allowed to push** to "No one", and **Allowed to merge** -to "Developers + Maintainers", to require everyone to submit a merge request for -changes going into the protected branch. This is compatible with workflows like -the [GitLab workflow](../../topics/gitlab_flow.md). - -However, there are workflows where that is not needed, and only protecting from -force pushes and branch removal is useful. For those workflows, you can allow -everyone with write access to push to a protected branch by setting -**Allowed to push** to "Developers + Maintainers". - -You can set the **Allowed to push** and **Allowed to merge** options while creating -a protected branch or afterwards by selecting the option you want from the -dropdown list in the **Already protected** area. - - - -If you don't choose any of those options while creating a protected branch, -they are set to Maintainers by default. - -### Allow deploy keys to push to a protected branch - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30769) in GitLab 13.7. -> - This feature is being selectively deployed in GitLab.com 13.7, and may not be available for all users. -> - This feature is available for all users in GitLab 13.9. - -You can allow specific machines to access protected branches in your repository with -[deploy keys](deploy_keys/index.md). This can be useful for your CI/CD workflow, -for example. - -Deploy keys can be selected in the **Allowed to push** dropdown when: +Prerequisite: -- Defining a protected branch. -- Updating an existing branch. +- You must have at least the [Maintainer role](../permissions.md). -Select a deploy key to allow the key's owner to push to the chosen protected branch, -even if they aren't a member of the related project. The owner of the selected deploy -key must have at least read access to the given project. +To protect multiple branches at the same time: -For a deploy key to be selectable: +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, type the branch name and a wildcard. + For example: -- It must be [enabled for your project](deploy_keys/index.md#how-to-enable-deploy-keys). -- It must have [write access](deploy_keys/index.md#deploy-keys-permissions) to your project repository. + | Wildcard protected branch | Matching branches | + |---------------------------|--------------------------------------------------------| + | `*-stable` | `production-stable`, `staging-stable` | + | `production/*` | `production/app-server`, `production/load-balancer` | + | `*gitlab*` | `gitlab`, `gitlab/staging`, `master/gitlab/production` | -Deploy keys are not available in the **Allowed to merge** dropdown. +1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. +1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. +1. Select **Protect**. - +The protected branch displays in the list of protected branches. -## Restricting push and merge access to certain users **(PREMIUM)** +## Create a protected branch -With GitLab Premium you can restrict access to protected branches -by choosing a role (Maintainers, Developers) and certain users. From the -dropdown menu select the role and/or the users you want to have merge or push -access. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53361) in GitLab 11.9. - +Users with the Developer or higher [role](../permissions.md) can create a protected branch. -Click **Protect** and the branch displays in the **Protected branch** list. +Prerequisites: - +- **Allowed to push** is set to **No one** +- **Allowed to merge** is set to **Developers**. -## Wildcard protected branches +You can create a protected branch by using the UI or API only. +This prevents you from accidentally creating a branch +from the command line or from a Git client application. -You can specify a wildcard protected branch, which protects all branches -matching the wildcard. For example: +To create a new branch through the user interface: -| Wildcard Protected Branch | Matching Branches | -|---------------------------|--------------------------------------------------------| -| `*-stable` | `production-stable`, `staging-stable` | -| `production/*` | `production/app-server`, `production/load-balancer` | -| `*gitlab*` | `gitlab`, `gitlab/staging`, `master/gitlab/production` | +1. Go to **Repository > Branches**. +1. Select **New branch**. +1. Fill in the branch name and select an existing branch, tag, or commit to + base the new branch on. Only existing protected branches and commits + that are already in protected branches are accepted. -Protected branch settings, like **Developers can push**, apply to all matching -branches. +## Require everyone to submit merge requests for a protected branch -Two different wildcards can potentially match the same branch. For example, -`*-stable` and `production-*` would both match a `production-stable` branch. -In that case, if _any_ of these protected branches have a setting like -"Allowed to push", then `production-stable` also inherit this setting. +You can force everyone to submit a merge request, rather than allowing them to check in directly +to a protected branch. This is compatible with workflows like the [GitLab workflow](../../topics/gitlab_flow.md). -If you click on a protected branch's name, GitLab displays a list of -all matching branches: +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to merge** list, select **Developers + Maintainers**. +1. From the **Allowed to push** list, select **No one**. +1. Select **Protect**. - +## Allow everyone to push directly to a protected branch -## Create a protected branch +You can allow everyone with write access to push to the protected branch. -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53361) in GitLab 11.9. +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to push** list, select **Developers + Maintainers**. +1. Select **Protect**. -When a protected branch or wildcard protected branches are set to -[**No one** is **Allowed to push**](#using-the-allowed-to-merge-and-allowed-to-push-settings), -Developers (and users with higher [permission levels](../permissions.md)) are -allowed to create a new protected branch as long as they are -[**Allowed to merge**](#using-the-allowed-to-merge-and-allowed-to-push-settings). -This can only be done by using the UI or through the API, to avoid creating protected -branches accidentally from the command line or from a Git client application. +## Allow deploy keys to push to a protected branch -To create a new branch through the user interface: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30769) in GitLab 13.7. +> - This feature was selectively deployed in GitLab.com 13.7, and may not be available for all users. +> - This feature is available for all users in GitLab 13.9. -1. Go to **Repository > Branches**. -1. Click on **New branch**. -1. Fill in the branch name and select an existing branch, tag, or commit to - base the new branch on. Only existing protected branches and commits - that are already in protected branches are accepted. +You can permit the owner of a [deploy key](deploy_keys/index.md) to push to a protected branch. +The deploy key works, even if the user isn't a member of the related project. However, the owner of the deploy +key must have at least read access to the project. -## Delete a protected branch +Prerequisites: -From time to time, you may need to delete or clean up protected branches. -User with the [Maintainer role](../permissions.md) and greater can manually delete protected -branches by using the GitLab web interface: +- The deploy key must be [enabled for your project](deploy_keys/index.md#how-to-enable-deploy-keys). +- The deploy key must have [write access](deploy_keys/index.md#deploy-keys-permissions) to your project repository. -1. Go to **Repository > Branches**. -1. Click on the delete icon next to the branch you wish to delete. -1. To prevent accidental deletion, an additional confirmation is required. +To allow a deploy key to push to a protected branch: -  +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to push** list, select the deploy key. +1. Select **Protect**. -Deleting a protected branch is allowed only by using the web interface; not from Git. -This means that you can't accidentally delete a protected branch from your -command line or a Git client application. +Deploy keys are not available in the **Allowed to merge** dropdown. -## Allow force push on protected branches +## Allow force push on a protected branch > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15611) in GitLab 13.10 behind a disabled feature flag. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323431) in GitLab 14.0. @@ -185,68 +153,74 @@ WARNING: This feature might not be available to you. Check the **version history** note above for details. You can allow [force pushes](../../topics/git/git_rebase.md#force-push) to -protected branches by either setting **Allowed to force push** -when you protect a new branch, or by configuring an already-protected branch. +protected branches. -To protect a new branch and enable Force push: +To protect a new branch and enable force push: -1. Navigate to your project's **Settings > Repository**. -1. Expand **Protected branches**, and scroll to **Protect a branch**. -1. Select a **Branch** or wildcard you'd like to protect. -1. Select the user levels **Allowed to merge** and **Allowed to push**. -1. To allow all users with push access to force push, toggle the **Allowed to force push** slider. -1. To reject code pushes that change files listed in the `CODEOWNERS` file, toggle - **Require approval from code owners**. -1. Click **Protect**. +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. +1. To allow all users with push access to force push, turn on the **Allowed to force push** toggle. +1. To reject code pushes that change files listed in the `CODEOWNERS` file, turn on the + **Require approval from code owners** toggle. +1. Select **Protect**. -To enable force pushes on branches already protected: +To enable force pushes on branches that are already protected: -1. Navigate to your project's **Settings > Repository**. -1. Expand **Protected branches** and scroll to **Protected branch**. -1. Toggle the **Allowed to force push** slider for the chosen branch. +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. In the list of protected branches, next to the branch, turn on the **Allowed to force push** toggle. -When enabled, members who are allowed to push to this branch can also force push. +When enabled, members who are can push to this branch can also force push. -## Protected branches approval by Code Owners **(PREMIUM)** +## Require Code Owner approval on a protected branch **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13251) in GitLab Premium 12.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13251) in GitLab Premium 12.4. +> - [In](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5 and later, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. -It is possible to require at least one approval by a -[Code Owner](code_owners.md) to a file changed by the -merge request. You can either set Code Owners approvals -at the time you protect a new branch, or set it to a branch -already protected, as described below. +You can require at least one approval by a [Code Owner](code_owners.md) to a file changed by the +merge request. To protect a new branch and enable Code Owner's approval: -1. Navigate to your project's **Settings > Repository** and expand **Protected branches**. -1. Scroll down to **Protect a branch**, select a **Branch** or wildcard you'd like to protect, select who's **Allowed to merge** and **Allowed to push**, and toggle the **Require approval from code owners** slider. -1. Click **Protect**. +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. +1. Turn on the **Require approval from code owners** toggle. +1. Select **Protect**. -To enable Code Owner's approval to branches already protected: +To enable Code Owner's approval on branches that are already protected: -1. Navigate to your project's **Settings > Repository** and expand **Protected branches**. -1. Scroll down to **Protected branch** and toggle the **Code owner approval** slider for the chosen branch. +1. Go to your project and select **Settings > Repository**. +1. Expand **Protected branches**. +1. In the list of protected branches, next to the branch, turn on the **Code owner approval** toggle. -When enabled, all merge requests targeting these branches require approval +When enabled, all merge requests for these branches require approval by a Code Owner per matched rule before they can be merged. Additionally, direct pushes to the protected branch are denied if a rule is matched. -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5, users and groups who are allowed to push to protected branches do not require a merge request to merge their feature branches. Thus, they can skip merge request approval rules. +## Run pipelines on protected branches -## Running pipelines on protected branches - -The permission to merge or push to protected branches is used to define if a user can -run CI/CD pipelines and execute actions on jobs that are related to those branches. +The permission to merge or push to protected branches defines +whether or not a user can run CI/CD pipelines and execute actions on jobs. See [Security on protected branches](../../ci/pipelines/index.md#pipeline-security-on-protected-branches) for details about the pipelines security model. -## Changelog +## Delete a protected branch + +Users with the [Maintainer role](../permissions.md) and greater can manually delete protected +branches by using the GitLab web interface: + +1. Go to **Repository > Branches**. +1. Next to the branch you want to delete, select the **Delete** button (**{remove}**). +1. On the confirmation dialog, type the branch name and select **Delete protected branch**. -- **13.5**: [Allow Deploy keys to push to protected branches once more](https://gitlab.com/gitlab-org/gitlab/-/issues/30769). -- **11.9**: [Allow protected branches to be created](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53361) - by Developers (and users with higher permission levels) through the API and the user interface. +You can delete a protected branch from the UI only. +This prevents you from accidentally deleting a branch +from the command line or from a Git client application. <!-- ## Troubleshooting diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 17a9cd5c8c6..b4e13aebdb2 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -9,15 +9,24 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10356) in GitLab 9.1. -Protected tags allow control over who has permission to create tags as well as preventing accidental update or deletion once created. Each rule allows you to match either an individual tag name, or use wildcards to control multiple tags at once. +Protected tags: + +- Allow control over who has permission to create tags. +- Prevent accidental update or deletion once created. + +Each rule allows you to match either: + +- An individual tag name. +- Wildcards to control multiple tags at once. This feature evolved out of [protected branches](protected_branches.md) -## Overview +## Who can modify a protected tag + +By default: -Protected tags prevent anyone from updating or deleting the tag, and prevent -creation of matching tags based on the permissions you have selected. By default, -anyone without Maintainer [permissions](../permissions.md) is prevented from creating tags. +- To create tags, you must have the [Maintainer role](../permissions.md). +- No one can update or delete tags. ## Configuring protected tags diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index 728c682b009..c17133e72cf 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -39,7 +39,7 @@ You can use push options to skip a CI/CD pipeline, or pass CI/CD variables. | Push option | Description | Introduced in version | | ------------------------------ | ------------------------------------------------------------------------------------------- |---------------------- | | `ci.skip` | Do not create a CI pipeline for the latest push. | [11.7](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15643) | -| `ci.variable="<name>=<value>"` | Provide [CI/CD variables](../../ci/variables/README.md) to be used in a CI pipeline, if one is created due to the push. | [12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/27983) | +| `ci.variable="<name>=<value>"` | Provide [CI/CD variables](../../ci/variables/index.md) to be used in a CI pipeline, if one is created due to the push. | [12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/27983) | An example of using `ci.skip`: @@ -66,6 +66,7 @@ time as pushing changes: | `merge_request.remove_source_branch` | Set the merge request to remove the source branch when it's merged. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.title="<title>"` | Set the title of the merge request. Ex: `git push -o merge_request.title="The title I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.description="<description>"` | Set the description of the merge request. Ex: `git push -o merge_request.description="The description I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | +| `merge_request.milestone="<milestone>"` | Set the milestone of the merge request. Ex: `git push -o merge_request.milestone="3.0"`. | [14.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63960) | | `merge_request.label="<label>"` | Add labels to the merge request. If the label does not exist, it is created. For example, for two labels: `git push -o merge_request.label="label1" -o merge_request.label="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.unlabel="<label>"` | Remove labels from the merge request. For example, for two labels: `git push -o merge_request.unlabel="label1" -o merge_request.unlabel="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.assign="<user>"` | Assign users to the merge request. For example, for two users: `git push -o merge_request.assign="user1" -o merge_request.assign="user2"`. | [13.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25904) | diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 45cb5e74d6c..d8d464ce6d8 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -69,11 +69,11 @@ threads. Some quick actions might not be available to all subscription tiers. | `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue and mark as a duplicate of another issue. **(FREE)** Also, mark both as related. | | `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. | | `/estimate <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. Learn more about [time tracking](time_tracking.md). | -| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants`. | +| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. | | `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced in GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/196795)). | | `/label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | | `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | -| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | +| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). | | `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. | | `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. | | `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/-/issues/10556)). | diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 71cbff9e545..3a25b148fdf 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -14,8 +14,11 @@ To introduce a checkpoint in your source code history, you can assign a However, in most cases, your users need more than just the raw source code. They need compiled objects or other assets output by your CI/CD system. -A GitLab *Release* is a snapshot of the source, build output, artifacts, and other metadata -associated with a released version of your code. +A GitLab Release can be: + +- A snapshot of the source code of your repository. +- [Generic packages](../../packages/generic_packages/index.md) created from job artifacts. +- Other metadata associated with a released version of your code. You can create a GitLab release on any branch. When you create a release: @@ -59,8 +62,8 @@ You can create a release in the user interface, or by using the We recommend using the API to create releases as one of the last steps in your CI/CD pipeline. -Only users with Developer permissions or higher can create releases. -Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). +Only users with at least the Developer role can create releases. +Read more about [Release permissions](#release-permissions). To create a new release through the GitLab UI: @@ -79,7 +82,7 @@ To create a new release through the GitLab UI: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/19298) in GitLab 12.7. -You can [create a release directly from the GitLab CI pipeline](../../../ci/yaml/README.md#release) +You can [create a release directly from the GitLab CI pipeline](../../../ci/yaml/index.md#release) by using a `release` node in the job definition. The release is created only if the job processes without error. If the Rails API returns an error @@ -99,8 +102,8 @@ release tag. When the `released_at` date and time has passed, the badge is autom > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26016) in GitLab 12.6. Asset link editing was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9427) in GitLab 12.10. -Only users with Developer permissions or higher can edit releases. -Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). +Only users with at least the Developer can edit releases. +Read more about [Release permissions](#release-permissions). To edit the details of a release: @@ -242,7 +245,7 @@ but are _not_ allowed to view details about the Git repository (in particular, tag names). Because of this, release titles are replaced with a generic title like "Release-1234" for Guest users to avoid leaking tag name information. -See the [Permissions](../../permissions.md#project-members-permissions) page for +See the [Release permissions](#release-permissions) section for more information about permissions. ### Tag name @@ -273,14 +276,28 @@ Release note descriptions are unrelated. Description supports [Markdown](../../m A release contains the following types of assets: - [Source code](#source-code) -- [Permanent links to release assets](#permanent-links-to-release-assets) +- [Link](#links) #### Source code GitLab automatically generates `zip`, `tar.gz`, `tar.bz2`, and `tar` archived source code from the given Git tag. These are read-only assets. -#### Permanent links to release assets +#### Links + +A link is any URL which can point to whatever you like: documentation, built +binaries, or other related materials. These can be both internal or external +links from your GitLab instance. +Each link as an asset has the following attributes: + +| Attribute | Description | Required | +| ---- | ----------- | --- | +| `name` | The name of the link. | Yes | +| `url` | The URL to download a file. | Yes | +| `filepath` | The redirect link to the `url`. See [this section](#permanent-links-to-release-assets) for more information. | No | +| `link_type` | The content kind of what users can download via `url`. See [this section](#link-types) for more information. | No | + +##### Permanent links to release assets > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27300) in GitLab 12.9. @@ -289,20 +306,6 @@ GitLab always redirects this URL to the actual asset location, so even if the assets move to a different location, you can continue to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link). -Each asset has a `name`, a `url` of the *actual* asset location, and optionally, -`filepath` and `link_type` parameters. - -A `filepath` creates a URL pointing to the asset for the Release. - -The `link_type` parameter accepts one of the following four values: - -- `runbook` -- `package` -- `image` -- `other` (default) - -This field has no effect on the URL and it's only used for visual purposes in the Releases page of your project. - The format of the URL is: ```plaintext @@ -329,13 +332,104 @@ https://gitlab.com/gitlab-org/gitlab-runner/releases/v11.9.0-rc2/downloads/binar The physical location of the asset can change at any time and the direct link remains unchanged. -### Links +##### Link Types -A link is any URL which can point to whatever you like: documentation, built -binaries, or other related materials. These can be both internal or external -links from your GitLab instance. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207257) in GitLab 13.1. The four types of links are "Runbook," "Package," "Image," and "Other." +The `link_type` parameter accepts one of the following four values: + +- `runbook` +- `package` +- `image` +- `other` (default) + +This field has no effect on the URL and it's only used for visual purposes in the Releases page of your project. + +##### Use a generic package for attaching binaries + +You can use [generic packages](../../packages/generic_packages/index.md) +to store any artifacts from a release or tag pipeline, +that can also be used for attaching binary files to an individual release entry. +You basically need to: + +1. [Push the artifacts to the Generic Package Registry](../../packages/generic_packages/index.md#publish-a-package-file). +1. [Attach the package link to the release](#links). + +The following example generates release assets, publishes them +as a generic package, and then creates a release: + +```yaml +stages: + - build + - upload + - release + +variables: + # Package version can only contain numbers (0-9), and dots (.). + # Must be in the format of X.Y.Z, i.e. should match /\A\d+\.\d+\.\d+\z/ regular expresion. + # See https://docs.gitlab.com/ee/user/packages/generic_packages/#publish-a-package-file + PACKAGE_VERSION: "1.2.3" + DARWIN_AMD64_BINARY: "myawesomerelease-darwin-amd64-${PACKAGE_VERSION}" + LINUX_AMD64_BINARY: "myawesomerelease-linux-amd64-${PACKAGE_VERSION}" + PACKAGE_REGISTRY_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/myawesomerelease/${PACKAGE_VERSION}" + +build: + stage: build + image: alpine:latest + rules: + - if: $CI_COMMIT_TAG + script: + - mkdir bin + - echo "Mock binary for ${DARWIN_AMD64_BINARY}" > bin/${DARWIN_AMD64_BINARY} + - echo "Mock binary for ${LINUX_AMD64_BINARY}" > bin/${LINUX_AMD64_BINARY} + artifacts: + paths: + - bin/ + +upload: + stage: upload + image: curlimages/curl:latest + rules: + - if: $CI_COMMIT_TAG + script: + - | + curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file bin/${DARWIN_AMD64_BINARY} ${PACKAGE_REGISTRY_URL}/${DARWIN_AMD64_BINARY} + - | + curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file bin/${LINUX_AMD64_BINARY} ${PACKAGE_REGISTRY_URL}/${LINUX_AMD64_BINARY} + +release: + # Caution, as of 2021-02-02 these assets links require a login, see: + # https://gitlab.com/gitlab-org/gitlab/-/issues/299384 + stage: release + image: registry.gitlab.com/gitlab-org/release-cli:latest + rules: + - if: $CI_COMMIT_TAG + script: + - | + release-cli create --name "Release $CI_COMMIT_TAG" --tag-name $CI_COMMIT_TAG \ + --assets-link "{\"name\":\"${DARWIN_AMD64_BINARY}\",\"url\":\"${PACKAGE_REGISTRY_URL}/${DARWIN_AMD64_BINARY}\"}" \ + --assets-link "{\"name\":\"${LINUX_AMD64_BINARY}\",\"url\":\"${PACKAGE_REGISTRY_URL}/${LINUX_AMD64_BINARY}\"}" +``` + +PowerShell users may need to escape the double quote `"` inside a JSON +string with a `` ` `` (back tick) for `--assets-link` and `ConvertTo-Json` +before passing on to the `release-cli`. +For example: + +```yaml +release: + script: + - $env:asset = "{`"name`":`"MyFooAsset`",`"url`":`"https://gitlab.com/upack/artifacts/download/$env:UPACK_GROUP/$env:UPACK_NAME/$($env:GitVersion_SemVer)?contentOnly=zip`"}" + - $env:assetjson = $env:asset | ConvertTo-Json + - release-cli create --name $CI_COMMIT_TAG --description "Release $CI_COMMIT_TAG" --ref $CI_COMMIT_TAG --tag-name $CI_COMMIT_TAG --assets-link=$env:assetjson +``` + +NOTE: +Directly attaching [job artifacts](../../../ci/pipelines/job_artifacts.md) +links to a release is not recommended, because artifacts are ephemeral and +are used to pass data in the same pipeline. This means there's a risk that +they could either expire or someone might manually delete them. ## Release evidence @@ -428,14 +522,14 @@ Evidence collection snapshots are visible on the Releases page, along with the t > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32773) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. -When you create a release, if [job artifacts](../../../ci/yaml/README.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. +When you create a release, if [job artifacts](../../../ci/yaml/index.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. Although job artifacts normally expire, artifacts included in release evidence do not expire. To enable job artifact collection you need to specify both: -1. [`artifacts:paths`](../../../ci/yaml/README.md#artifactspaths) -1. [`artifacts:reports`](../../../ci/yaml/README.md#artifactsreports) +1. [`artifacts:paths`](../../../ci/yaml/index.md#artifactspaths) +1. [`artifacts:reports`](../../../ci/yaml/index.md#artifactsreports) ```yaml ruby: @@ -455,7 +549,7 @@ release evidence. If you [schedule release evidence collection](#schedule-release-evidence-collection), some artifacts may already be expired by the time of evidence collection. To avoid this you can use -the [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) +the [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) keyword. Learn more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). ### Schedule release evidence collection @@ -471,6 +565,49 @@ In the API: - If you do not specify a `released_at` date, release evidence is collected on the date the release is created. +## Release permissions + +> [The permission model for create, update and delete actions was fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/327505) in GitLab 14.1. + +### View a release and download assets + +- Users with [Reporter role or above](../../../user/permissions.md#project-members-permissions) + have read and download access to the project releases. +- Users with [Guest role](../../../user/permissions.md#project-members-permissions) + have read and download access to the project releases, however, + repository-related information are redacted (for example the Git tag name). + +### Create, update, and delete a release and its assets + +- Users with [Developer role or above](../../../user/permissions.md#project-members-permissions) + have write access to the project releases and assets. +- If a release is associated with a [protected tag](../protected_tags.md), + the user must be [allowed to create the protected tag](../protected_tags.md#configuring-protected-tags) too. + +As an example of release permission control, you can allow only +[Maintainer role or above](../../../user/permissions.md#project-members-permissions) +to create, update, and delete releases by protecting the tag with a wildcard (`*`), +and set **Maintainer** in the **Allowed to create** column. + +#### Enable or disable protected tag evaluation on releases **(FREE SELF)** + +Protected tag evaluation on release permissions is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:evalute_protected_tag_for_release_permissions) +``` + +To disable it: + +```ruby +Feature.disable(:evalute_protected_tag_for_release_permissions) +``` + ## Release Command Line > [Introduced](https://gitlab.com/gitlab-org/release-cli/-/merge_requests/6) in GitLab 12.10. @@ -494,14 +631,13 @@ These metrics include: - Total number of releases in the group - Percentage of projects in the group that have at least one release -<!-- ## Troubleshooting +## Troubleshooting + +### Getting `403 Forbidden` or `Something went wrong while creating a new release` errors when creating, updating or deleting releases and their assets -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +If the release is associted with a [protected tag](../protected_tags.md), +the UI/API request might result in an authorization failure. +Make sure that the user or a service/bot account is allowed to +[create the protected tag](../protected_tags.md#configuring-protected-tags) too. -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +See [the release permissions](#release-permissions) for more information. diff --git a/doc/user/project/repository/branches/default.md b/doc/user/project/repository/branches/default.md index 6c2469ac377..2f1171a7d4f 100644 --- a/doc/user/project/repository/branches/default.md +++ b/doc/user/project/repository/branches/default.md @@ -57,9 +57,7 @@ GitLab administrators can configure a new default branch name at the ### Instance-level custom initial branch name **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221013) in GitLab 13.2. -> - It's deployed behind a feature flag, enabled by default. -> - It cannot be enabled or disabled per-project. -> - It's recommended for production use. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/325163) in GitLab 13.12. GitLab [administrators](../../../permissions.md) of self-managed instances can customize the initial branch for projects hosted on that instance. Individual @@ -154,8 +152,45 @@ renames a Git repository's (`example`) default branch. 1. Update references to the old branch name in related code and scripts that reside outside your repository, such as helper utilities and integrations. +## Default branch rename redirect + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329100) in GitLab 14.1 + +URLs for specific files or directories in a project embed the project's default +branch name, and are often found in documentation or browser bookmarks. When you +[update the default branch name in your repository](#update-the-default-branch-name-in-your-repository), +these URLs change, and must be updated. + +To ease the transition period, whenever the default branch for a project is +changed, GitLab records the name of the old default branch. If that branch is +deleted, attempts to view a file or directory on it are redirected to the +current default branch, instead of displaying the "not found" page. + ## Resources - [Discussion of default branch renaming](https://lore.kernel.org/git/pull.656.v4.git.1593009996.gitgitgadget@gmail.com/) on the Git mailing list - [March 2021 blog post: The new Git default branch name](https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/) + +## Troubleshooting + +### Unable to change default branch: resets to current branch + +We are tracking this problem in [issue 20474](https://gitlab.com/gitlab-org/gitlab/-/issues/20474). +This issue often occurs when a branch named `HEAD` is present in the repository. +To fix the problem: + +1. In your local repository, create a new, temporary branch and push it: + + ```shell + git checkout -b tmp_default && git push -u origin tmp_default + ``` + +1. In GitLab, proceed to [change the default branch](#change-the-default-branch-name-for-a-project) to that temporary branch. +1. From your local repository, delete the `HEAD` branch: + + ```shell + git push -d origin HEAD + ``` + +1. In GitLab, [change the default branch](#change-the-default-branch-name-for-a-project) to the one you intend to use. diff --git a/doc/user/project/repository/csv.md b/doc/user/project/repository/csv.md new file mode 100644 index 00000000000..4bf6c7451d5 --- /dev/null +++ b/doc/user/project/repository/csv.md @@ -0,0 +1,24 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# CSV files **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14174) in GitLab 14.1. + +A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. +Each line of the file is a data record. Each record consists of one or more fields, separated by +commas. The use of the comma as a field separator is the source of the name for this file format. +A CSV file typically stores tabular data (numbers and text) in plain text, in which case each line +will have the same number of fields. + +The CSV file format is not fully standardized. Other characters can be used as column delimiters. +Fields may or may not be surrounded to escape special characters. + +When added to a repository, files with a `.csv` extension are rendered as a table when viewed in +GitLab. + + diff --git a/doc/user/project/repository/img/csv_file_rendered_as_table_v14_1.png b/doc/user/project/repository/img/csv_file_rendered_as_table_v14_1.png Binary files differnew file mode 100644 index 00000000000..0f9b623e7b4 --- /dev/null +++ b/doc/user/project/repository/img/csv_file_rendered_as_table_v14_1.png diff --git a/doc/user/project/repository/img/repository_mirroring_pull_settings_lower.png b/doc/user/project/repository/img/repository_mirroring_pull_settings_lower.png Binary files differdeleted file mode 100644 index a3e0b74ddf8..00000000000 --- a/doc/user/project/repository/img/repository_mirroring_pull_settings_lower.png +++ /dev/null diff --git a/doc/user/project/repository/img/repository_mirroring_pull_settings_upper.png b/doc/user/project/repository/img/repository_mirroring_pull_settings_upper.png Binary files differdeleted file mode 100644 index 8e15b5a0784..00000000000 --- a/doc/user/project/repository/img/repository_mirroring_pull_settings_upper.png +++ /dev/null diff --git a/doc/user/project/repository/img/repository_mirroring_push_settings.png b/doc/user/project/repository/img/repository_mirroring_push_settings.png Binary files differdeleted file mode 100644 index 8916d21d515..00000000000 --- a/doc/user/project/repository/img/repository_mirroring_push_settings.png +++ /dev/null diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 7919850b8cc..afdcf2a94fa 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -38,10 +38,10 @@ to a branch in the repository. When you use the command line, you can commit mul In GitLab, you can add keywords to the commit message to perform one of the following actions: - **Trigger a GitLab CI/CD pipeline:** - If the project is configured with [GitLab CI/CD](../../../ci/README.md), + If the project is configured with [GitLab CI/CD](../../../ci/index.md), you trigger a pipeline per push, not per commit. - **Skip pipelines:** - Add the [`ci skip`](../../../ci/yaml/README.md#skip-pipeline) keyword to + Add the [`ci skip`](../../../ci/yaml/index.md#skip-pipeline) keyword to your commit message to make GitLab CI/CD skip the pipeline. - **Cross-link issues and merge requests:** Use [cross-linking](../issues/crosslinking_issues.md#from-commit-messages) diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index e9f214f08ce..71bece03a33 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -11,13 +11,16 @@ Repository mirroring allows for the mirroring of repositories to and from extern can use it to mirror branches, tags, and commits between repositories. It helps you use a repository outside of GitLab. -A repository mirror at GitLab updates automatically. You can also manually trigger an update -at most once every five minutes on GitLab.com with [the limit set by the administrator on self-managed instances](../../../administration/instance_limits.md#pull-mirroring-interval). +A repository mirror at GitLab updates automatically. You can also manually trigger an update: + +- At most once every five minutes on GitLab.com. +- According to a [limit set by the administrator](../../../administration/instance_limits.md#pull-mirroring-interval) + on self-managed instances. There are two kinds of repository mirroring supported by GitLab: -- [Push](#pushing-to-a-remote-repository): for mirroring a GitLab repository to another location. **(FREE)** -- [Pull](#pulling-from-a-remote-repository): for mirroring a repository from another location to GitLab. **(PREMIUM)** +- [Push](#push-to-a-remote-repository): for mirroring a GitLab repository to another location. +- [Pull](#pull-from-a-remote-repository): for mirroring a repository from another location to GitLab. When the mirror repository is updated, all new branches, tags, and commits are visible in the project's activity feed. @@ -48,9 +51,9 @@ The following are some possible use cases for repository mirroring: GitLab.com repository that's public, allows you to open source specific projects and contribute back to the open source community. -## Pushing to a remote repository **(FREE)** +## Push to a remote repository -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40137) in GitLab 13.5: LFS support over HTTPS. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40137) in GitLab 13.5: LFS support over HTTPS. For an existing project, you can set up push mirroring as follows: @@ -63,8 +66,6 @@ For an existing project, you can set up push mirroring as follows: 1. Select the **Keep divergent refs** check box, if desired. 1. Select **Mirror repository** to save the configuration. - - When push mirroring is enabled, only push commits directly to the mirrored repository to prevent the mirror diverging. @@ -72,7 +73,7 @@ Unlike [pull mirroring](#how-it-works), the mirrored repository is not periodica The mirrored repository receives all changes only when: - Commits are pushed to GitLab. -- A [forced update](#forcing-an-update) is initiated. +- A [forced update](#force-an-update) is initiated. Changes pushed to files in the repository are automatically pushed to the remote mirror at least: @@ -82,14 +83,14 @@ Changes pushed to files in the repository are automatically pushed to the remote In the case of a diverged branch, an error displays in the **Mirroring repositories** section. -### Configuring push mirrors through the API +### Configure push mirrors through the API You can also create and modify project push mirrors through the [remote mirrors API](../../../api/remote_mirrors.md). ### Keep divergent refs -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208828) in GitLab 13.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208828) in GitLab 13.0. By default, if any ref on the remote mirror has diverged from the local repository, the *entire push* fails, and no updates occur. @@ -108,11 +109,11 @@ update. NOTE: After the mirror is created, this option can only be modified via the [API](../../../api/remote_mirrors.md). -### Setting up a push mirror from GitLab to GitHub +### Set up a push mirror from GitLab to GitHub To set up a mirror from GitLab to GitHub, you need to follow these steps: -1. Create a [GitHub personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with the `public_repo` box checked. +1. Create a [GitHub personal access token](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) with the `public_repo` box checked. 1. Fill in the **Git repository URL** field using this format: `https://<your_github_username>@github.com/<your_github_group>/<your_github_project>.git`. 1. Fill in **Password** field with your GitHub personal access token. 1. Select **Mirror repository**. @@ -121,7 +122,7 @@ The mirrored repository is listed. For example, `https://*****:*****@github.com/ The repository pushes shortly thereafter. To force a push, select the **Update now** (**{retry}**) button. -### Setting up a push mirror from GitLab to AWS CodeCommit +### Set up a push mirror from GitLab to AWS CodeCommit AWS CodeCommit push mirroring is the best way to connect GitLab repositories to AWS CodePipeline, as GitLab isn't yet supported as one of their Source Code Management (SCM) providers. @@ -203,7 +204,7 @@ To test mirroring by forcing a push, select the half-circle arrows button (hover If **Last successful update** shows a date, you have configured mirroring correctly. If it isn't working correctly, a red `error` tag appears and shows the error message as hover text. -### Setting up a push mirror to another GitLab instance with 2FA activated +### Set up a push mirror to another GitLab instance with 2FA activated 1. On the destination GitLab instance, create a [personal access token](../../profile/personal_access_tokens.md) with `write_repository` scope. 1. On the source GitLab instance: @@ -211,7 +212,7 @@ If it isn't working correctly, a red `error` tag appears and shows the error mes 1. Fill in the **Password** field with the GitLab personal access token created on the destination GitLab instance. 1. Select **Mirror repository**. -## Pulling from a remote repository **(PREMIUM)** +## Pull from a remote repository **(PREMIUM)** > - [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab/-/issues/10871) in GitLab 11.11. > - Moved to GitLab Premium in 13.9. @@ -224,7 +225,7 @@ to browse its content and its activity using the GitLab interface, you can confi mirror pulling: 1. If you [configured two-factor authentication (2FA)](https://docs.github.com/en/github/authenticating-to-github/securing-your-account-with-two-factor-authentication-2fa) - for GitHub, create a [personal access token for GitHub](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) + for GitHub, create a [personal access token for GitHub](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) with the `read_repository` scope. If 2FA is enabled, this personal access token serves as your GitHub password. 1. In your project, go to **Settings > Repository**, and then expand the @@ -238,18 +239,12 @@ mirror pulling: - **Only mirror protected branches** 1. Select **Mirror repository** to save the configuration. - - ---- - - - Because GitLab is now set to pull changes from the upstream repository, you should not push commits directly to the repository on GitLab. Instead, any commits should be pushed to the remote repository. Changes pushed to the remote repository are pulled into the GitLab repository, either: - Automatically in a certain period of time. -- When a [forced update](#forcing-an-update) is initiated. +- When a [forced update](#force-an-update) is initiated. WARNING: If you do manually update a branch in the GitLab repository, the branch becomes diverged from @@ -273,7 +268,7 @@ Repository mirrors are updated as Sidekiq becomes available to process them. If ### Overwrite diverged branches **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. You can choose to always update your local branches with remote versions, even if they have diverged from the remote. @@ -285,7 +280,7 @@ To use this option, check the **Overwrite diverged branches** box when creating ### Trigger pipelines for mirror updates **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. If this option is enabled, pipelines trigger when branches or tags are updated from the remote repository. Depending on the activity of the remote @@ -295,7 +290,7 @@ assigned when you set up pull mirroring. ### Hard failure **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. After 14 consecutive unsuccessful retries, the mirroring process is marked as a hard failure and mirroring attempts stop. This failure is visible in either the: @@ -303,11 +298,11 @@ and mirroring attempts stop. This failure is visible in either the: - Project's main dashboard. - Pull mirror settings page. -You can resume the project mirroring again by [forcing an update](#forcing-an-update). +You can resume the project mirroring again by [forcing an update](#force-an-update). ### Trigger an update using the API **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. Pull mirroring uses polling to detect new branches and commits added upstream, often minutes afterwards. If you notify GitLab by [API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project), @@ -317,18 +312,19 @@ For more information, see [Start the pull mirroring process for a Project](../.. ## Mirror only protected branches **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. Based on the mirror direction that you choose, you can opt to mirror only the -[protected branches](../protected_branches.md) from/to your remote repository. -For pull mirroring, non-protected branches are not mirrored and can diverge. +[protected branches](../protected_branches.md) in the mirroring project, +either from or to your remote repository. For pull mirroring, non-protected branches in +the mirroring project are not mirrored and can diverge. To use this option, check the **Only mirror protected branches** box when creating a repository mirror. **(PREMIUM)** ## SSH authentication -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22982) in GitLab 11.6 for Push mirroring. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22982) in GitLab 11.6 for Push mirroring. SSH authentication is mutual: @@ -369,7 +365,7 @@ fingerprints in the open for you to check: - [AWS CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints) - [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/) -- [GitHub](https://docs.github.com/en/github/authenticating-to-github/githubs-ssh-key-fingerprints) +- [GitHub](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints) - [GitLab.com](../../gitlab_com/index.md#ssh-host-keys-fingerprints) - [Launchpad](https://help.launchpad.net/SSHFingerprints) - [Savannah](http://savannah.gnu.org/maintenance/SshAccess/) @@ -417,7 +413,7 @@ NOTE: The generated keys are stored in the GitLab database, not in the file system. Therefore, SSH public key authentication for mirrors cannot be used in a pre-receive hook. -## Forcing an update **(FREE)** +## Force an update **(FREE)** While mirrors are scheduled to update automatically, you can always force an update by using the update button which is available on the **Mirroring repositories** section of the **Repository Settings** page. @@ -426,7 +422,7 @@ update button which is available on the **Mirroring repositories** section of th ## Bidirectional mirroring **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. WARNING: Bidirectional mirroring may cause conflicts. @@ -450,13 +446,17 @@ protected branches. ### Configure a webhook to trigger an immediate pull to GitLab -Assuming you have already configured the [push](#setting-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) and [pull](#pulling-from-a-remote-repository) mirrors in the upstream GitLab instance, to trigger an immediate pull as suggested above, you must configure a [Push Event Web Hook](../integrations/webhooks.md#push-events) in the downstream instance. +Assuming you have already configured the [push](#set-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) +and [pull](#pull-from-a-remote-repository) mirrors in the upstream GitLab instance, to trigger an +immediate pull as suggested above, you must configure a [Push Event Web Hook](../integrations/webhooks.md#push-events) +in the downstream instance. To do this: 1. Create a [personal access token](../../profile/personal_access_tokens.md) with `API` scope. 1. In your project, go to **Settings > Webhooks**. -1. Add the webhook URL which (in this case) uses the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) request to trigger an immediate pull after updates to the repository. +1. Add the webhook URL which (in this case) uses the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) + request to trigger an immediate pull after updates to the repository. ```plaintext https://gitlab.example.com/api/v4/projects/:id/mirror/pull?private_token=<your_access_token> @@ -467,7 +467,7 @@ To do this: To test the integration, select the **Test** button and confirm GitLab doesn't return an error message. -### Preventing conflicts using a `pre-receive` hook +### Prevent conflicts using a pre-receive hook WARNING: The solution proposed negatively affects the performance of @@ -550,7 +550,7 @@ Note that this sample has a few limitations: - The script circumvents the Git hook quarantine environment because the update of `$TARGET_REPO` is seen as a ref update, and Git displays warnings about it. -### Mirroring with Perforce Helix via Git Fusion **(PREMIUM)** +### Mirror with Perforce Helix via Git Fusion **(PREMIUM)** > Moved to GitLab Premium in 13.9. @@ -583,14 +583,52 @@ Should an error occur during a push, GitLab displays an **Error** highlight for ### 13:Received RST_STREAM with error code 2 with GitHub -If you receive an "13:Received RST_STREAM with error code 2" while mirroring to a GitHub repository, your GitHub settings might be set to block pushes that expose your email address used in commits. Either set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting. +If you receive a "13:Received RST_STREAM with error code 2" message while mirroring to a GitHub repository, +your GitHub settings might be set to block pushes that expose your email address used in commits. Either +set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting. ### 4:Deadline Exceeded -When upgrading to GitLab 11.11.8 or newer, a change in how usernames are represented means that you may need to update your mirroring username and password to ensure that `%40` characters are replaced with `@`. +When upgrading to GitLab 11.11.8 or newer, a change in how usernames are represented means that you +may need to update your mirroring username and password to ensure that `%40` characters are replaced with `@`. ### Connection blocked because server only allows public key authentication -As the error indicates, the connection is getting blocked between GitLab and the remote repository. Even if a [TCP Check](../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site) is successful, you must check any networking components in the route from GitLab to the remote Server to ensure there's no blockage. +As the error indicates, the connection is getting blocked between GitLab and the remote repository. Even if a +[TCP Check](../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site) is successful, +you must check any networking components in the route from GitLab to the remote Server to ensure there's no blockage. For example, we've seen this error when a Firewall was performing a `Deep SSH Inspection` on outgoing packets. + +### Could not read username: terminal prompts disabled + +If you receive this error after creating a new project using +[GitLab CI/CD for external repositories](../../../ci/ci_cd_for_external_repos/): + +```plaintext +"2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': terminal prompts disabled\n": exit status 128." +``` + +Check if the repository owner is specified in the URL of your mirrored repository: + +1. Go to your project. +1. In the left sidebar, select **Settings > Repository**. +1. Select **Mirroring repositories**. +1. If no repository owner is specified, delete and add the URL again in this format: + + ```plaintext + https://**<repo_owner>**@bitbucket.org/<accountname>/<reponame>.git + ``` + +The repository owner is needed for Bitbucket to connect to the repository for mirroring. + +### Pull mirror is missing LFS files + +In some cases, pull mirroring does not transfer LFS files. This issue occurs when: + +- You use an SSH repository URL. The workaround is to use an HTTPS repository URL instead. + There is [an issue to fix this for SSH URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/11997). +- You're using GitLab 14.0 or older, and the source repository is a public Bitbucket URL. + This was [fixed in GitLab 14.0.6](https://gitlab.com/gitlab-org/gitlab/-/issues/335123). +- You mirror an external repository using object storage. + There is [an issue to fix this](https://gitlab.com/gitlab-org/gitlab/-/issues/335495). diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index d7fbff23e5e..4ac1113152c 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -122,7 +122,7 @@ You can also sort the requirements list by: > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/215514) ability to specify individual requirements and their statuses in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. -GitLab supports [requirements test reports](../../../ci/yaml/README.md#artifactsreportsrequirements) now. +GitLab supports [requirements test reports](../../../ci/yaml/index.md#artifactsreportsrequirements) now. You can add a job to your CI pipeline that, when triggered, marks all existing requirements as Satisfied (you may manually satisfy a requirement in the edit form [edit a requirement](#edit-a-requirement)). diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index dd646a54b43..d46e55ca005 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -147,7 +147,7 @@ To use a custom issue template with Service Desk, in your project: 1. Go to **Settings > General > Service Desk**. 1. From the dropdown **Template to append to all Service Desk issues**, select your template. -### Using custom email display name +### Using a custom email display name > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7529) in GitLab 12.8. @@ -160,22 +160,29 @@ To edit the custom email display name: 1. Enter a new name in **Email display name**. 1. Select **Save Changes**. -### Using custom email address **(FREE SELF)** +### Using a custom email address **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab Premium 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284656) in GitLab 13.8. -If the `service_desk_email` is configured, then you can create Service Desk -issues by sending emails to the Service Desk email address. The default -address has the following format: -`project_contact+%{key}@example.com`. +It is possible to customize the email address used by Service Desk. To do this, you must configure +both a [custom mailbox](#configuring-a-custom-mailbox) and a +[custom suffix](#configuring-a-custom-email-address-suffix). -The `%{key}` part is used to find the project where the issue should be created. The -`%{key}` part combines the path to the project and configurable project name suffix: -`<project_full_path>-<project_name_suffix>`. +#### Configuring a custom mailbox -You can set the project name suffix in your project's Service Desk settings. -It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). +NOTE: +On GitLab.com a custom mailbox is already configured with `contact-project+%{key}@incoming.gitlab.com` as the email address, so you only have to configure the +[custom suffix](#configuring-a-custom-email-address-suffix) in project settings. + +Using the `service_desk_email` configuration, you can customize the mailbox +used by Service Desk. This allows you to have a separate email address for +Service Desk by also configuring a [custom suffix](#configuring-a-custom-email-address-suffix) +in project settings. + +The `address` must include the `+%{key}` placeholder within the 'user' +portion of the address, before the `@`. This is used to identify the project +where the issue should be created. NOTE: The `service_desk_email` and `incoming_email` configurations should @@ -183,7 +190,7 @@ always use separate mailboxes. This is important, because emails picked from `service_desk_email` mailbox are processed by a different worker and it would not recognize `incoming_email` emails. -To configure a custom email address for Service Desk with IMAP, add the following snippets to your configuration file: +To configure a custom mailbox for Service Desk with IMAP, add the following snippets to your configuration file in full: - Example for installations from source: @@ -207,57 +214,37 @@ To configure a custom email address for Service Desk with IMAP, add the followin ```ruby gitlab_rails['service_desk_email_enabled'] = true - gitlab_rails['service_desk_email_address'] = "project_contact+%{key}@gmail.com" - gitlab_rails['service_desk_email_email'] = "project_support@gmail.com" - gitlab_rails['service_desk_email_password'] = "[REDACTED]" - gitlab_rails['service_desk_email_mailbox_name'] = "inbox" - gitlab_rails['service_desk_email_idle_timeout'] = 60 - gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" - gitlab_rails['service_desk_email_host'] = "imap.gmail.com" - gitlab_rails['service_desk_email_port'] = 993 - gitlab_rails['service_desk_email_ssl'] = true - gitlab_rails['service_desk_email_start_tls'] = false ``` -In this case, suppose the `mygroup/myproject` project Service Desk settings has the project name -suffix set to `support`, and a user sends an email to `project_contact+mygroup-myproject-support@example.com`. -As a result, a new Service Desk issue is created from this email in the `mygroup/myproject` project. - The configuration options are the same as for configuring [incoming email](../../administration/incoming_email.md#set-it-up). -#### Microsoft Graph +##### Microsoft Graph > Introduced in [GitLab 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/214900) Service Desk can be configured to read Microsoft Exchange Online mailboxes with the Microsoft -Graph API instead of IMAP. Follow the [documentation in the incoming e-mail section for setting up an OAuth2 application for Microsoft Graph](../../administration/incoming_email.md#microsoft-graph). +Graph API instead of IMAP. Follow the [documentation in the incoming email section for setting up an OAuth2 application for Microsoft Graph](../../administration/incoming_email.md#microsoft-graph). - Example for Omnibus GitLab installations: ```ruby gitlab_rails['service_desk_email_enabled'] = true - gitlab_rails['service_desk_email_address'] = "project_contact+%{key}@example.onmicrosoft.com" - gitlab_rails['service_desk_email_email'] = "project_contact@example.onmicrosoft.com" - gitlab_rails['service_desk_email_mailbox_name'] = "inbox" - gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" - gitlab_rails['service_desk_inbox_method'] = 'microsoft_graph' - gitlab_rails['service_desk_inbox_options'] = { 'tenant_id': '<YOUR-TENANT-ID>', 'client_id': '<YOUR-CLIENT-ID>', @@ -268,6 +255,22 @@ Graph API instead of IMAP. Follow the [documentation in the incoming e-mail sect The Microsoft Graph API is not yet supported in source installations. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/326169) for more details. +#### Configuring a custom email address suffix + +You can set a custom suffix in your project's Service Desk settings once you have configured a [custom mailbox](#configuring-a-custom-mailbox). +It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). + +When configured, the custom suffix creates a new Service Desk email address, consisting of the +`service_desk_email_address` setting and a key of the format: `<project_full_path>-<custom_suffix>` + +For example, suppose the `mygroup/myproject` project Service Desk settings has the following configured: + +- Project name suffix is set to `support`. +- Service Desk email address is configured to `contact+%{key}@example.com`. + +The Service Desk email address for this project is: `contact+mygroup-myproject-support@example.com`. +The [incoming email](../../administration/incoming_email.md) address still works. + ## Using Service Desk You can use Service Desk to [create an issue](#as-an-end-user-issue-creator) or [respond to one](#as-a-responder-to-the-issue). diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 890784cecf5..d5fdb86c9b0 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -110,7 +110,7 @@ and the exports between them are compatible. You can export projects from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. This assumes [version history](#version-history) requirements are met. -If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../README.md). +If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../index.md). ## Exported contents @@ -156,7 +156,7 @@ To export a project and its data, follow these steps:  -1. Once the export is generated, you should receive an e-mail with a link to +1. Once the export is generated, you should receive an email with a link to download the file:  @@ -183,7 +183,7 @@ To export a project and its data, follow these steps: NOTE: If use of the `Internal` visibility level -[is restricted](../../../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects), +[is restricted](../../../public_access/public_access.md#restrict-use-of-public-or-internal-projects), all imported projects are given the visibility of `Private`. NOTE: diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 03a77e42765..d79d9f5b9dc 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -165,7 +165,7 @@ cannot change them: - Includes any jobs that drive the logic of your job. - Explicitly set the container image file to run the job in. This ensures that your script steps execute in the correct environment. -- Explicitly set any relevant GitLab pre-defined [job keywords](../../../ci/yaml/README.md#job-keywords). +- Explicitly set any relevant GitLab pre-defined [job keywords](../../../ci/yaml/index.md#job-keywords). This ensures that your job uses the settings you intend and that they are not overriden by project-level pipelines. @@ -188,8 +188,8 @@ Use the switches to enable or disable the following features: | **Issues** | ✓ | Activates the GitLab issues tracker | | **Repository** | ✓ | Enables [repository](../repository/) functionality | | **Merge Requests** | ✓ | Enables [merge request](../merge_requests/) functionality; also see [Merge request settings](#merge-request-settings) | -| **Forks** | ✓ | Enables [forking](../working_with_projects.md#fork-a-project) functionality | -| **Pipelines** | ✓ | Enables [CI/CD](../../../ci/README.md) functionality | +| **Forks** | ✓ | Enables [forking](../repository/forking_workflow.md) functionality | +| **Pipelines** | ✓ | Enables [CI/CD](../../../ci/index.md) functionality | | **Container Registry** | | Activates a [registry](../../packages/container_registry/) for your Docker images | | **Git Large File Storage** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs) | | **Packages** | | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration) functionality | @@ -251,7 +251,7 @@ Set up your project's merge request settings: - Enable [merge request approvals](../merge_requests/approvals/index.md). - Enable [status checks](../merge_requests/status_checks.md). - Enable [merge only if pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). -- Enable [merge only when all threads are resolved](../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-threads-are-resolved). +- Enable [merge only when all threads are resolved](../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved). - Enable [require an associated issue from Jira](../../../integration/jira/issues.md#require-associated-jira-issue-for-merge-requests-to-be-merged). - Enable [`delete source branch after merge` option by default](../merge_requests/getting_started.md#deleting-the-source-branch). - Configure [suggested changes commit messages](../merge_requests/reviews/suggestions.md#configure-the-commit-message-for-applied-suggestions). @@ -267,7 +267,8 @@ Learn how to [export a project](import_export.md#importing-the-project) in GitLa ### Advanced settings -Here you can run housekeeping, archive, rename, transfer, [remove a fork relationship](#removing-a-fork-relationship), or remove a project. +Here you can run housekeeping, archive, rename, transfer, +[remove a fork relationship](#removing-a-fork-relationship), or delete a project. #### Archiving a project @@ -363,28 +364,54 @@ namespace if needed. #### Delete a project -NOTE: -Only project Owners and administrators have [permissions](../../permissions.md#project-members-permissions) to delete a project. +You can mark a project to be deleted. + +Prerequisite: + +- You must have at least the Owner role for a project. To delete a project: -1. Navigate to your project, and select **Settings > General > Advanced**. -1. In the "Delete project" section, click the **Delete project** button. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. In the "Delete project" section, select **Delete project**. 1. Confirm the action when asked to. -This action: +This action deletes a project including all associated resources (issues, merge requests, and so on). -- Deletes a project including all associated resources (issues, merge requests etc). -- From [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) on [Premium](https://about.gitlab.com/pricing/) or higher tiers, - group Owners can [configure](../../group/index.md#enable-delayed-project-removal) projects within a group - to be deleted after a delayed period. - When enabled, actual deletion happens after number of days - specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). +In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) and later, on Premium or higher tiers, +group Owners can [configure](../../group/index.md#enable-delayed-project-removal) projects in a group +to be deleted after a delayed period. +When enabled, actual deletion happens after number of days +specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). WARNING: -The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to +The default behavior of [delayed project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to [Immediate deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. +#### Delete a project immediately **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/191367) in GitLab 14.1. + +If you don't want to wait, you can delete a project immediately. + +Prerequisites: + +- You must have at least the Owner role for a project. +- You have [marked the project for deletion](#delete-a-project). + +To immediately delete a project marked for deletion: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. In the "Permanently delete project" section, select **Delete project**. +1. Confirm the action when asked to. + +Your project, its repository, and all related resources, including issues and merge requests, +are deleted. + #### Restore a project **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6. diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index d7121239610..5e045ee2455 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -8,20 +8,24 @@ type: reference, howto # Project access tokens NOTE: -Project access tokens are supported for self-managed instances on Free and above. They are also supported on GitLab SaaS Premium and above (excluding [trial licenses](https://about.gitlab.com/free-trial/)). +Project access tokens are supported for self-managed instances on Free and above. They are also supported on GitLab SaaS Premium and above (excluding [trial licenses](https://about.gitlab.com/free-trial/)). Self-managed Free instances should review their security and compliance policies with regards to [user self-enrollment](../../admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups) and consider [disabling project access tokens](#enable-or-disable-project-access-token-creation) to lower potential abuse. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2587) in GitLab 13.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210181) in GitLab 13.0. > - [Became available on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/235765) in GitLab 13.5 for paid groups only. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235765) in GitLab 13.5. WARNING: This feature might not be available to you. Check the **version history** note above for details. -Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). You can also use project access tokens with Git to authenticate over HTTP. +Project access tokens are scoped to a project and can be used to authenticate with the +[GitLab API](../../../api/index.md#personalproject-access-tokens). You can also use +project access tokens with Git to authenticate over HTTPS. If you are asked for a +username when authenticating over HTTPS, you can use any non-empty value because only +the token is needed. Project access tokens expire on the date you define, at midnight UTC. -For examples of how you can use a project access token to authenticate with the API, see the following section from our [API Docs](../../../api/README.md#personalproject-access-tokens). +For examples of how you can use a project access token to authenticate with the API, see the following section from our [API Docs](../../../api/index.md#personalproject-access-tokens). ## Creating a project access token @@ -29,17 +33,21 @@ For examples of how you can use a project access token to authenticate with the 1. Navigate to the project you would like to create an access token for. 1. In the **Settings** menu choose **Access Tokens**. 1. Choose a name and optional expiry date for the token. +1. Choose a role for the token. 1. Choose the [desired scopes](#limiting-scopes-of-a-project-access-token). 1. Click the **Create project access token** button. 1. Save the project access token somewhere safe. Once you leave or refresh - the page, you won't be able to access it again. + the page, you don't have access to it again. ## Project bot users +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210181) in GitLab 13.0. +> - [Excluded from license seat use](https://gitlab.com/gitlab-org/gitlab/-/issues/223695) in GitLab 13.5. + Project bot users are [GitLab-created service accounts](../../../subscriptions/self_managed/index.md#billable-users) and do not count as licensed seats. For each project access token created, a bot user is created and added to the project with -[Maintainer level permissions](../../permissions.md#project-members-permissions). +the [specified level permissions](../../permissions.md#project-members-permissions). For the bot: diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index b7fd14ae74b..29aedb33003 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -6,16 +6,12 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Time Tracking +# Time tracking **(FREE)** -> Introduced in GitLab 8.14. +With time tracking you can track estimates and time spent on issues and merge +requests in GitLab. -Time Tracking allows you to track estimates and time spent on issues and merge -requests within GitLab. - -## Overview - -Time Tracking allows you to: +Use time tracking for these tasks: - Record the time spent working on an issue or a merge request. - Add an estimate of the amount of time needed to complete an issue or a merge @@ -24,14 +20,13 @@ Time Tracking allows you to: You don't have to indicate an estimate to enter the time spent, and vice versa. -Data about time tracking is shown on the issue/merge request sidebar, as shown -below. +Data about time tracking shows up on the issue and merge request sidebar:  ## How to enter data -Time Tracking uses two [quick actions](quick_actions.md): `/spend` and `/estimate`. +Time tracking uses two [quick actions](quick_actions.md): `/spend` and `/estimate`. If you use either quick action more than once in a single comment, only the last occurrence is applied. @@ -44,9 +39,10 @@ with [Reporter and higher permission levels](../permissions.md). ### Estimates -To enter an estimate, write `/estimate`, followed by the time. For example, if -you need to enter an estimate of 1 month, 2 weeks, 3 days, 4 hours and 5 minutes, -write `/estimate 1mo 2w 3d 4h 5m`. +To enter an estimate, type `/estimate`, followed by the time. + +For example, if you need to enter an estimate of 1 month, 2 weeks, 3 days, 4 hours, and 5 minutes, +type `/estimate 1mo 2w 3d 4h 5m`. Check the [time units you can use](#configuration). Every time you enter a new time estimate, any previous time estimates are @@ -57,21 +53,22 @@ To remove an estimation entirely, use `/remove_estimate`. ### Time spent -To enter time spent, write `/spend`, followed by the time. For example, if you need -to log 1 month, 2 weeks, 3 days, 4 hours and 5 minutes, you would write `/spend 1mo 2w 3d 4h 5m`. -Time units that we support are listed at the bottom of this help page. +To enter time spent, type `/spend`, followed by the time. + +For example, if you need +to log 1 month, 2 weeks, 3 days, 4 hours, and 5 minutes, type `/spend 1mo 2w 3d 4h 5m`. +Check the [time units you can use](#configuration). Every new time spent entry is added to the current total time spent for the issue or the merge request. -You can remove time by entering a negative amount: for example, `/spend -3d` removes three +To subtract time, enter a negative value. For example, `/spend -3d` removes three days from the total time spent. You can't go below 0 minutes of time spent, -so GitLab automatically resets the time spent if you remove a larger amount -of time compared to the time that was entered already. +so if you remove more time than already entered, GitLab ignores the subtraction. You can log time in the past by providing a date after the time. For example, if you want to log 1 hour of time spent on the 31 January 2021, -you would write `/spend 1h 2021-01-31`. If you supply a date in the future, the +you would type `/spend 1h 2021-01-31`. If you supply a date in the future, the command fails and no time is logged. To remove all the time spent at once, use `/remove_time_spent`. @@ -97,13 +94,13 @@ The breakdown of spent time is limited to a maximum of 100 entries. The following time units are available: -- Months (mo) -- Weeks (w) -- Days (d) -- Hours (h) -- Minutes (m) - -Default conversion rates are 1mo = 4w, 1w = 5d and 1d = 8h. +| Time unit | What to type | Default conversion rate | +| --------- | ------------ | ----------------------- | +| Month | `mo` | 4w | +| Week | `w` | 5d | +| Day | `d` | 8h | +| Hour | `h` | 60m | +| Minute | `m` | | ### Limit displayed units to hours **(FREE SELF)** @@ -121,11 +118,11 @@ To do so: With this option enabled, `75h` is displayed instead of `1w 4d 3h`. -## Other interesting links +## Related links -- [Time Tracking solutions page](https://about.gitlab.com/solutions/time-tracking/) -- Time Tracking GraphQL references: +- [Time tracking solutions page](https://about.gitlab.com/solutions/time-tracking/) +- Time tracking GraphQL references: - [Connection](../../api/graphql/reference/index.md#timelogconnection) - [Edge](../../api/graphql/reference/index.md#timelogedge) - [Fields](../../api/graphql/reference/index.md#timelog) - - [Group Timelogs](../../api/graphql/reference/index.md#grouptimelogs) + - [Group timelogs](../../api/graphql/reference/index.md#grouptimelogs) diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 0e597725611..722505304c0 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -331,7 +331,7 @@ The [File Sync](#file-syncing-to-web-terminal) feature is supported on Kubernete In order to enable the Web IDE terminals you need to create the file `.gitlab/.gitlab-webide.yml` inside the repository's root. This -file is fairly similar to the [CI configuration file](../../../ci/yaml/README.md) +file is fairly similar to the [CI configuration file](../../../ci/yaml/index.md) syntax but with some restrictions: - No global blocks (such as `before_script` or `after_script`) can be defined. diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index ed6a51665bd..527db38c980 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -44,7 +44,7 @@ users see when viewing the wiki: ## Create a new wiki page -Users with Developer [permissions](../../permissions.md) can create new wiki pages: +Users with the [Developer role](../../permissions.md) can create new wiki pages: 1. Go to your project or group and select **Wiki**. 1. Select **New page** on this page, or any other wiki page. @@ -108,7 +108,7 @@ may not be able to check out the wiki locally afterward. ## Edit a wiki page -You need Developer [permissions](../../permissions.md) or higher to edit a wiki page: +You need the [Developer role](../../permissions.md) or higher to edit a wiki page: 1. Go to your project or group and select **Wiki**. 1. Go to the page you want to edit. @@ -133,7 +133,7 @@ You need the [Maintainer role](../../permissions.md) or higher to delete a wiki ## Move a wiki page -You need Developer [permissions](../../permissions.md) or higher to move a wiki page: +You need the [Developer role](../../permissions.md) or higher to move a wiki page: 1. Go to your project or group and select **Wiki**. 1. Go to the page you want to move. @@ -234,7 +234,7 @@ wikis, with a few limitations: For updates, follow [the epic that tracks feature parity with project wikis](https://gitlab.com/groups/gitlab-org/-/epics/2782). -Group wikis can be edited by members with [Developer permissions](../../permissions.md#group-members-permissions) +Group wikis can be edited by members with the [Developer role](../../permissions.md#group-members-permissions) and above. Group wiki repositories can be moved using the [Group repository storage moves API](../../../api/group_repository_storage_moves.md). diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index a0b20f5c86d..08b63bfed5f 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -14,12 +14,17 @@ code are saved in projects, and most features are in the scope of projects. You can explore other popular projects available on GitLab. To explore projects: 1. On the top bar, select **Menu > Project**. -1. Select **Explore Projects**. +1. Select **Explore projects**. GitLab displays a list of projects, sorted by last updated date. To view projects with the most [stars](#star-a-project), click **Most stars**. To view projects with the largest number of comments in the past month, click **Trending**. +NOTE: +By default, `/explore` is visible to unauthenticated users. However, if the +[**Public** visibility level](../admin_area/settings/visibility_and_access_controls.md#restricted-visibility-levels) +is restricted, `/explore` is visible only to signed-in users. + ## Create a project To create a project in GitLab: @@ -84,7 +89,7 @@ Built-in templates are project templates that are: - Developed and maintained in the [`project-templates`](https://gitlab.com/gitlab-org/project-templates) and [`pages`](https://gitlab.com/pages) groups. - Released with GitLab. -- Anyone can contribute a built-in template by following [these steps](https://about.gitlab.com/community/contribute/project-templates). +- Anyone can contribute a built-in template by following [these steps](https://about.gitlab.com/community/contribute/project-templates/). To use a built-in template on the **New project** page: @@ -148,7 +153,7 @@ and then [cloning the repository](../../gitlab-basics/start-using-git.md#clone-a locally, you can directly push it to GitLab to create the new project, all without leaving your terminal. If you have access rights to the associated namespace, GitLab automatically creates a new project under that GitLab namespace with its visibility -set to Private by default (you can later change it in the [project's settings](../../public_access/public_access.md#how-to-change-project-visibility)). +set to Private by default (you can later change it in the [project's settings](../../public_access/public_access.md#change-project-visibility)). This can be done by using either SSH or HTTPS: diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md index 20a2a7263c3..bb74a035121 100644 --- a/doc/user/search/advanced_search.md +++ b/doc/user/search/advanced_search.md @@ -12,6 +12,7 @@ type: reference NOTE: This is the user documentation. To configure the Advanced Search, visit the [administrator documentation](../../integration/elasticsearch.md). +Advanced Search is enabled in GitLab.com. GitLab Advanced Search expands on the Basic Search with an additional set of features for faster, more advanced searches across the entire GitLab instance @@ -34,6 +35,11 @@ The Advanced Search can be useful in various scenarios: Advanced Search is based on Elasticsearch, which is a purpose-built full text search engine that can be horizontally scaled so that it can provide search results in 1-2 seconds in most cases. +- **Code Maintenance:** + Finding all the code that needs to be updated at once across an entire + instance can save time spent maintaining code. + This is especially helpful for organizations with more than 10 active projects. + This can also help build confidence is code refactoring to identify unknown impacts. - **Promote innersourcing:** Your company may consist of many different developer teams each of which has their own group where the various projects are hosted. Some of your applications diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 0cdaa3150c5..d90841fae88 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -17,9 +17,9 @@ and to-do items are assigned to you:  -- **(issues)** **Issues**: The open issues assigned to you. -- **(merge-request-open)** **Merge requests**: The [merge requests](../project/merge_requests/index.md) assigned to you. -- **(todo-done)** **To-do items**: The [to-do items](../todos.md) assigned to you. +- **{issues}** **Issues**: The open issues assigned to you. +- **{merge-request-open}** **Merge requests**: The [merge requests](../project/merge_requests/index.md) assigned to you. +- **{todo-done}** **To-do items**: The [to-do items](../todos.md) assigned to you. When you click **Issues**, GitLab shows the opened issues assigned to you: @@ -39,9 +39,9 @@ in the search field in the upper right corner: ### Filtering issue and merge request lists -> - Filtering by Epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. -> - Filtering by child Epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. -> - Filtering by Iterations was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6. [Moved](../../subscriptions/bronze_starter.md) to GitLab Premium in 13.9. +> - Filtering by Epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in GitLab Ultimate 12.9. +> - Filtering by child Epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in GitLab Ultimate 13.0. +> - Filtering by Iterations was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6. Moved to GitLab Premium in 13.9. Follow these steps to filter the **Issues** and **Merge Requests** list pages in projects and groups: @@ -107,7 +107,7 @@ You can filter the **Issues** list to individual instances by their ID. For exam ### Filtering merge requests by approvers **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/9468) in GitLab 11.9. -> - [Moved](../../subscriptions/bronze_starter.md) to GitLab Premium in 13.9. +> - Moved to GitLab Premium in 13.9. To filter merge requests by an individual approver, you can type (or select from the dropdown) **Approver** and select the user. @@ -117,7 +117,7 @@ the dropdown) **Approver** and select the user. ### Filtering merge requests by "approved by" **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30335) in GitLab 13.0. -> - [Moved](../../subscriptions/bronze_starter.md) to GitLab Premium in 13.9. +> - Moved to GitLab Premium in 13.9. To filter merge requests already approved by a specific individual, you can type (or select from the dropdown) **Approved-By** and select the user. diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 4b3f9e78c7b..9fb6152168e 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -128,9 +128,9 @@ A single snippet can support up to 10 files, which helps keep related files toge If you need more than 10 files for your snippet, we recommend you create a [wiki](project/wiki/index.md) instead. Wikis are available for projects at all subscription levels, and [groups](project/wiki/index.md#group-wikis) for -[GitLab Premium](https://about.gitlab.com/pricing). +[GitLab Premium](https://about.gitlab.com/pricing/). -Snippets with multiple files display a file count in the [snippet list](http://snippets.gitlab.com/): +Snippets with multiple files display a file count in the [snippet list](https://gitlab.com/dashboard/snippets):  diff --git a/doc/user/todos.md b/doc/user/todos.md index 4227f46dfa8..719ab6f6e7c 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -52,11 +52,11 @@ A to-do item appears on your To-Do List when: pipeline succeeds. - [In GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12136) and later, a merge request is removed from a - [merge train](../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md), + [merge train](../ci/pipelines/merge_trains.md), and you're the user that added it. When several trigger actions occur for the same user on the same object (for -example, an issue), GitLab displays only the first action as a single to do +example, an issue), GitLab displays only the first action as a single to-do item. To-do item triggers aren't affected by [GitLab notification email settings](profile/notifications.md). @@ -99,7 +99,7 @@ You can also add the following to your To-Do List by clicking the **Add a to do* - [Epics](group/epics/index.md) - [Designs](project/issues/design_management.md) - + ## Marking a to-do item as done @@ -129,12 +129,12 @@ If no action is needed, you can manually mark the to-do item as done by clicking its corresponding **Done** button to have GitLab remove the item from your To-Do List. - + You can also mark a to-do item as done by clicking the **Mark as done** button in the sidebar of an issue, merge request, or epic. - + You can mark all your to-do items as done at once by clicking the **Mark all as done** button. @@ -147,9 +147,9 @@ You can use the following types of filters with your To-Do List: | ------- | ---------------------------------------------------------------- | | Project | Filter by project. | | Group | Filter by group. | -| Author | Filter by the author that triggered the to do. | +| Author | Filter by the author that triggered the to-do item. | | Type | Filter by issue, merge request, design, or epic. | -| Action | Filter by the action that triggered the to do. | +| Action | Filter by the action that triggered the to-do item. | You can also filter by more than one of these at the same time. The previously described [triggering actions](#what-triggers-a-to-do-item) include: diff --git a/doc/user/workspace/img/1.1-Instance_overview.png b/doc/user/workspace/img/1.1-Instance_overview.png Binary files differnew file mode 100644 index 00000000000..7612cc7c092 --- /dev/null +++ b/doc/user/workspace/img/1.1-Instance_overview.png diff --git a/doc/user/workspace/img/1.2-Groups_overview.png b/doc/user/workspace/img/1.2-Groups_overview.png Binary files differnew file mode 100644 index 00000000000..b4f034bba16 --- /dev/null +++ b/doc/user/workspace/img/1.2-Groups_overview.png diff --git a/doc/user/workspace/img/1.3-Admin.png b/doc/user/workspace/img/1.3-Admin.png Binary files differnew file mode 100644 index 00000000000..018ed8a1bfc --- /dev/null +++ b/doc/user/workspace/img/1.3-Admin.png diff --git a/doc/user/workspace/img/Admin_Settings.png b/doc/user/workspace/img/Admin_Settings.png Binary files differnew file mode 100644 index 00000000000..5c4656ff342 --- /dev/null +++ b/doc/user/workspace/img/Admin_Settings.png diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md new file mode 100644 index 00000000000..f098cef6053 --- /dev/null +++ b/doc/user/workspace/index.md @@ -0,0 +1,34 @@ +--- +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Workspace + +Workspace will be the top-level [namespace](../group/index.md#namespaces) for you to manage +everything GitLab, including: + +- Defining and applying settings to all of your groups, subgroups, and projects. +- Aggregating data from all your groups, subgroups, and projects. + +Workspace will take many of the features from the +[Admin Area](../admin_area/index.md), and there will be one workspace per: + +- Instance, for self-managed instances. +- Namespace, for GitLab.com. + +NOTE: +Workspace is currently in development. + +## Concept previews + +The following provide a preview to the Workspace concept. + + + + + + + + |
