diff options
Diffstat (limited to 'doc/user')
264 files changed, 5014 insertions, 4203 deletions
diff --git a/doc/user/admin_area/activating_deactivating_users.md b/doc/user/admin_area/activating_deactivating_users.md index 06a26737495..448c65038c2 100644 --- a/doc/user/admin_area/activating_deactivating_users.md +++ b/doc/user/admin_area/activating_deactivating_users.md @@ -39,7 +39,7 @@ A user can be deactivated from the Admin Area. To do this: Please note that for the deactivation option to be visible to an admin, the user: - Must be currently active. -- Must not have any signins or activity in the last 180 days. +- Must not have signed in, or have any activity, in the last 180 days. Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 5eecfbb73c6..9259c93cfa3 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -18,9 +18,11 @@ Using Credentials inventory, GitLab administrators can see all the personal acce - Who they belong to. - Their access scope. - Their usage pattern. +- When they expire. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. +- When they were revoked. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. To access the Credentials inventory, navigate to **Admin Area > Credentials**. The following is an example of the Credentials inventory page: - + diff --git a/doc/user/admin_area/img/credentials_inventory_v12_6.png b/doc/user/admin_area/img/credentials_inventory_v12_6.png Binary files differdeleted file mode 100644 index 5c16781cb2d..00000000000 --- a/doc/user/admin_area/img/credentials_inventory_v12_6.png +++ /dev/null diff --git a/doc/user/admin_area/img/credentials_inventory_v13_2.png b/doc/user/admin_area/img/credentials_inventory_v13_2.png Binary files differnew file mode 100644 index 00000000000..5b56422a0a3 --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_v13_2.png diff --git a/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png b/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png Binary files differnew file mode 100644 index 00000000000..04d01f2662f --- /dev/null +++ b/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png diff --git a/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png b/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png Binary files differnew file mode 100644 index 00000000000..c6ca2bac83c --- /dev/null +++ b/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 1ffaf4e0678..c5e29612596 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Activate all GitLab Enterprise Edition functionality with a license **(STARTER ONLY)** +# Activate GitLab EE with a license **(STARTER ONLY)** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload a license. Once you've received your license from GitLab Inc., you can upload it @@ -107,14 +107,23 @@ expired license(s). It's possible to upload and view more than one license, but only the latest license will be used as the active license. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### There is no License tab in the Admin Area -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +If you originally installed Community Edition rather than Enterprise Edition you will need to +[upgrade to Enterprise Edition](../../update/README.md#community-to-enterprise-edition) +before uploading your license. + +GitLab.com users cannot upload and use a self-managed license. If you +wish to use paid features on GitLab.com, a separate subscription may be +[purchased](../../subscriptions/index.md#subscribe-to-gitlabcom). + +### Users exceed license limit upon renewal + +If you've added new users to your GitLab instance prior to renewal you may need to +purchase additional seats to cover those users. If this is the case and a license +without enough users is uploaded a message is displayed prompting you to purchase +additional users. More information on how to determine the required number of users +and how to add additional seats can be found in the +[licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). diff --git a/doc/user/admin_area/merge_requests_approvals.md b/doc/user/admin_area/merge_requests_approvals.md index 153ccfc128a..6d9d634ce14 100644 --- a/doc/user/admin_area/merge_requests_approvals.md +++ b/doc/user/admin_area/merge_requests_approvals.md @@ -34,3 +34,22 @@ Merge request approval rules that can be set at an instance level are: - **Prevent users from modifying merge request approvers list**. Prevents project maintainers from allowing users to modify the approvers list in project settings or in individual merge requests. + +## Scope rules to compliance-labeled projects + +> Introduced in [GitLab Premium](https://gitlab.com/groups/gitlab-org/-/epics/3432) 13.2. + +Merge request approval rules can be further scoped to specific compliance frameworks. + +When the compliance framework label is selected and the project is assigned the compliance +label, the instance-level MR approval settings will take effect and the +[project-level settings](../project/merge_requests/merge_request_approvals.md#adding--editing-a-default-approval-rule) +is locked for modification. + +When the compliance framework label is not selected or the project is not assigned the +compliance label, the project-level MR approval settings will take effect and the users with +Maintainer role and above can modify these. + +| Instance-level | Project-level | +| -------------- | ------------- | +|  |  | diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 91e29118e3e..329b6ff5bb0 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -137,8 +137,8 @@ This check is being exempt from Rack Attack. ## Access token (Deprecated) -> NOTE: **Note:** -> Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). +NOTE: **Note:** +Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the **Admin Area > Monitoring > Health check** @@ -152,6 +152,10 @@ The access token can be passed as a URL parameter: https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` +NOTE: **Note:** +In case the database or Redis service are unaccessible, the probe endpoints response is not guaranteed to be correct. +You should switch to [IP whitelist](#ip-whitelist) from deprecated access token to avoid it. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index b4fb7a524da..167016f1cb5 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -30,11 +30,12 @@ details. This sets a maximum size limit on each namespace. The following are included in the namespace size: -- repository -- wiki +- Repository +- Wiki - LFS objects -- build artifacts -- packages +- Build artifacts +- Packages +- Snippets NOTE: **Note:** This limit is not currently enforced but will be in a future release. @@ -140,6 +141,39 @@ Once a lifetime for personal access tokens is set, GitLab will: allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, or remove it, before revocation takes place. +## Optional enforcement of Personal Access Token expiry **(ULTIMATE ONLY)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214723) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - It is deployed behind a feature flag, disabled by default. +> - It is disabled on GitLab.com. +> - It is not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-optional-enforcement-of-personal-access-token-expiry-feature-core-only). **(CORE ONLY)** + +GitLab administrators can choose to prevent personal access tokens from expiring automatically. The tokens will be usable after the expiry date, unless they are revoked explicitly. + +To do this: + +1. Navigate to **Admin Area > Settings > General**. +1. Expand the **Account and limit** section. +1. Uncheck the **Enforce personal access token expiration** checkbox. + +### Enable or disable optional enforcement of Personal Access Token expiry Feature **(CORE ONLY)** + +Optional Enforcement of Personal Access Token Expiry is under development and not ready for production use. It is deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can enable it for your instance from the [rails console](../../../administration/feature_flags.md#start-the-gitlab-rails-console). + +To enable it: + +```ruby +Feature.enable(:enforce_personal_access_token_expiration) +``` + +To disable it: + +```ruby +Feature.disable(:enforce_personal_access_token_expiration) +``` + ## Disabling user profile name changes **(PREMIUM ONLY)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24605) in GitLab 12.7. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 3a287f29a0a..0479da7fb52 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -17,8 +17,8 @@ You can find it in the **Admin Area > Settings > CI/CD**. To enable (or disable) [Auto DevOps](../../../topics/autodevops/index.md) for all projects: -1. Go to **Admin Area > Settings > CI/CD** -1. Check (or uncheck to disable) the box that says "Default to Auto DevOps pipeline for all projects" +1. Go to **Admin Area > Settings > CI/CD**. +1. Check (or uncheck to disable) the box that says **Default to Auto DevOps pipeline for all projects**. 1. Optionally, set up the [Auto DevOps base domain](../../../topics/autodevops/index.md#auto-devops-base-domain) which is going to be used for Auto Deploy and Auto Review Apps. 1. Hit **Save changes** for the changes to take effect. @@ -48,21 +48,21 @@ To change it at the: 1. Go to **Admin Area > Settings > CI/CD**. 1. Change the value of maximum artifacts size (in MB). - 1. Hit **Save changes** for the changes to take effect. + 1. Click **Save changes** for the changes to take effect. - [Group level](../../group/index.md#group-settings) (this will override the instance setting): 1. Go to the group's **Settings > CI / CD > General Pipelines**. 1. Change the value of **maximum artifacts size (in MB)**. - 1. Press **Save changes** for the changes to take effect. + 1. Click **Save changes** for the changes to take effect. - [Project level](../../../ci/pipelines/settings.md) (this will override the instance and group settings): 1. Go to the project's **Settings > CI / CD > General Pipelines**. 1. Change the value of **maximum artifacts size (in MB)**. - 1. Press **Save changes** for the changes to take effect. + 1. Click **Save changes** for the changes to take effect. -NOTE: **Note** +NOTE: **Note:** The setting at all levels is only available to GitLab administrators. ## Default artifacts expiration **(CORE ONLY)** @@ -70,18 +70,17 @@ The setting at all levels is only available to GitLab administrators. The default expiration time of the [job artifacts](../../../administration/job_artifacts.md) can be set in the Admin Area of your GitLab instance. The syntax of duration is described in [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) -and the default value is `30 days`. On GitLab.com they -[never expire](../../gitlab_com/index.md#gitlab-cicd). +and the default value is `30 days`. 1. Go to **Admin Area > Settings > CI/CD**. 1. Change the value of default expiration time. -1. Hit **Save changes** for the changes to take effect. +1. Click **Save changes** for the changes to take effect. This setting is set per job and can be overridden in [`.gitlab-ci.yml`](../../../ci/yaml/README.md#artifactsexpire_in). To disable the expiration, set it to `0`. The default unit is in seconds. -NOTE: **Note** +NOTE: **Note:** Any changes to this setting will apply to new artifacts only. The expiration time will not be updated for artifacts created before this setting was changed. The administrator may need to manually search for and expire previously-created @@ -101,9 +100,10 @@ On GitLab.com, the quota is calculated based on your To change the pipelines minutes quota: -1. Go to **Admin Area > Settings > CI/CD** -1. Set the pipeline minutes quota limit. -1. Hit **Save changes** for the changes to take effect +1. Go to **Admin Area > Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. In the **Pipeline minutes quota** box, enter the maximum number of minutes. +1. Click **Save changes** for the changes to take effect. --- @@ -112,8 +112,8 @@ also change each group's pipeline minutes quota to override the global value. 1. Navigate to the **Admin Area > Overview > Groups** and hit the **Edit** button for the group you wish to change the pipeline minutes quota. -1. Set the pipeline minutes quota to the desired value -1. Hit **Save changes** for the changes to take effect. +1. In the **Pipeline Minutes Quota** box, enter the maximum number of minutes. +1. Click **Save changes** for the changes to take effect. Once saved, you can see the build quota in the group admin view. The quota can also be viewed in the project admin view if shared Runners @@ -143,6 +143,8 @@ Once that time passes, the jobs will be archived and no longer able to be retried. Make it empty to never expire jobs. It has to be no less than 1 day, for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. +As of June 22, 2020 the [value is set](../../gitlab_com/index.md#gitlab-cicd) to 3 months on GitLab.com. Jobs created before that date will be archived after September 22, 2020. + ## Default CI configuration path > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18073) in GitLab 12.5. diff --git a/doc/user/admin_area/settings/img/email_notification_for_unknown_sign_ins_v13_2.png b/doc/user/admin_area/settings/img/email_notification_for_unknown_sign_ins_v13_2.png Binary files differnew file mode 100644 index 00000000000..fdcc542c4d7 --- /dev/null +++ b/doc/user/admin_area/settings/img/email_notification_for_unknown_sign_ins_v13_2.png diff --git a/doc/user/admin_area/settings/img/import_export_rate_limits_v13_2.png b/doc/user/admin_area/settings/img/import_export_rate_limits_v13_2.png Binary files differnew file mode 100644 index 00000000000..78c94b3803c --- /dev/null +++ b/doc/user/admin_area/settings/img/import_export_rate_limits_v13_2.png diff --git a/doc/user/admin_area/settings/import_export_rate_limits.md b/doc/user/admin_area/settings/import_export_rate_limits.md new file mode 100644 index 00000000000..92cb2a1a109 --- /dev/null +++ b/doc/user/admin_area/settings/import_export_rate_limits.md @@ -0,0 +1,32 @@ +--- +type: reference +stage: Manage +group: Import +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Project/Group Import/Export rate limits + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35728) in GitLab 13.2. + +The following table includes configurable rate limits. The following table includes limits on a +per minute per user basis: + +| Limit | Default (per minute per user) | +|--------------------------|-------------------------------| +| Project Import | 6 | +| Project Export | 6 | +| Project Export Download | 1 | +| Group Import | 6 | +| Group Export | 6 | +| Group Export Download | 1 | + +All rate limits are: + +- Configurable at **(admin)** **Admin Area > Settings > Network > Import/Export Rate Limits** +- Applied per minute per user +- Not applied per IP address +- Active by default. To disable, set the option to `0` +- Logged to `auth.log` file if exceed rate limit + + diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index df087722fcf..8c1e82f838b 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -43,6 +43,7 @@ Access the default page for admin area settings by navigating to | Option | Description | | ------ | ----------- | +| [Repository's custom initial branch name](../../project/repository/branches/index.md#custom-initial-branch-name-core-only) | Set a custom branch name rather than master for all the new repositories created within your instance. | | [Repository mirror](visibility_and_access_controls.md#allow-mirrors-to-be-set-up-for-projects) | Configure repository mirroring. | | [Repository storage](../../../administration/repository_storage_types.md) | Configure storage path settings. | | Repository maintenance | ([Repository checks](../../../administration/repository_checks.md) and [Housekeeping](../../../administration/housekeeping.md)). Configure automatic Git checks and housekeeping on repositories. | diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md index bae60aba15f..3f8ef7c9d01 100644 --- a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -10,10 +10,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28129) in GitLab 12.10. This setting allows you to rate limit the requests to the issue creation endpoint. -It defaults to 300 requests per minute. -You can change it in **Admin Area > Settings > Network > Performance Optimization**. +You can change its value in **Admin Area > Settings > Network > Issues Rate Limits**. -For example, requests using the +For example, if you set a limit of 300, requests using the [Projects::IssuesController#create](https://gitlab.com/gitlab-org/gitlab/raw/master/app/controllers/projects/issues_controller.rb) action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. @@ -23,6 +22,6 @@ This limit is: - Applied independently per project and per user. - Not applied per IP address. -- Active by default. To disable it, set the option to `0`. +- Disabled by default. To enable it, set the option to any value other than `0`. Requests over the rate limit are logged into the `auth.log` file. diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index 1da93c7005f..311b79af7e3 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -4,9 +4,14 @@ type: reference # Sign-in restrictions **(CORE ONLY)** -You can use sign-in restrictions to limit the authentication with password -for web interface and Git over HTTP(S), two-factor authentication enforcing, as well as -as configuring the home page URL and after sign-out path. +You can use **Sign-in restrictions** to customize authentication restrictions for web interfaces as well as Git over HTTP(S). + +## Settings + +To access sign-in restriction settings: + +1. Navigate to the **Admin Area > Settings > General**. +1. Expand the **Sign-in restrictions** section. ## Password authentication enabled @@ -25,6 +30,15 @@ period in hours.  +## Email notification for unknown sign-ins + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218457) in GitLab 13.2. + +When enabled, GitLab notifies users of sign-ins from unknown IP addresses or devices. For more information, +see [Email notification for unknown sign-ins](../../profile/unknown_sign_in_notification.md). + + + ## Sign-in information All users that are not logged-in will be redirected to the page represented by the configured @@ -36,13 +50,6 @@ after sign out if value is not empty. If a "Sign in text" in Markdown format is provided, then every user will be presented with this message after logging-in. -## Settings - -To access this feature: - -1. Navigate to the **Admin Area > Settings > General**. -1. Expand the **Sign-in restrictions** section. - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index d9ca4a0881a..92eeb6a04b7 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -68,8 +68,16 @@ To ensure only admin users can delete projects: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6. -By default, a project or group marked for removal will be permanently removed after 7 days. -This period may be changed, and setting this period to 0 will enable immediate removal +By default, a project marked for deletion will be permanently removed with immediate effect. +By default, a group marked for deletion will be permanently removed after 7 days. + +CAUTION: **Warning:** +The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to +[Immediate deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. + +Projects within a group can be deleted after a delayed period, by [configuring in Group Settings](../../group/index.md#enabling-delayed-project-removal-premium). + +The default period is 7 days, and can be changed. Setting this period to 0 will enable immediate removal of projects or groups. To change this period: diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 0efe28ac5f7..b11bae98af3 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -332,12 +332,6 @@ The current permissions on the Project Value Stream Analytics dashboard are: You can [read more about permissions](../../ci/yaml/README.md) in general. -NOTE: **Note:** -As of GitLab 12.3, the project-level page is deprecated. You should access -project-level Value Stream Analytics from **Analytics > Value Stream Analytics** in the top -navigation bar. We will ensure that the same project-level functionality is available -to CE users in the new analytics space. - For Value Stream Analytics functionality introduced in GitLab 12.3 and later: - Users must have Reporter access or above. diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index f0fcd0c4419..229a8572206 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -14,17 +14,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w The security configuration page displays the configuration state of each of the security features and can be accessed through a project's sidebar nav. - + The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines/index.md) to determine the configuration state of each feature. If a job with the expected security report artifact exists in the pipeline, the feature is considered configured. -NOTE: **Note:** if the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), +NOTE: **Note:** +If the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), all security features will be configured by default. ## Limitations -It is not possible to enable or disable a feature using the configuration page. -However, instructions on how to enable or disable a feature can be found through -the links next to each feature on that page. +It is not yet possible to enable or disable most features using the +configuration page. However, instructions on how to enable or disable a feature +can be found through the links next to each feature on that page. + +If a project does not have an existing CI configuration, then the SAST feature +can be enabled by clicking on the "Enable with Merge Request" button under the +"Manage" column. Future work will expand this to editing _existing_ CI +configurations, and to other security features. diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png b/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png Binary files differdeleted file mode 100644 index 7a079a65072..00000000000 --- a/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v13_2.png b/doc/user/application_security/container_scanning/img/container_scanning_v13_2.png Binary files differnew file mode 100644 index 00000000000..254ea1dcf5d --- /dev/null +++ b/doc/user/application_security/container_scanning/img/container_scanning_v13_2.png diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 0ffe83cdfc9..7bc8b62825c 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -32,7 +32,7 @@ You can enable container scanning by doing one of the following: GitLab compares the found vulnerabilities between the source and target branches, and shows the information directly in the merge request. - + <!-- NOTE: The container scanning tool references the following heading in the code, so if you make a change to this heading, make sure to update the documentation URLs used in the @@ -58,10 +58,10 @@ To enable Container Scanning in your pipeline, you need the following: ```yaml build: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: @@ -114,7 +114,7 @@ build: image: docker:stable stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: @@ -141,7 +141,7 @@ enables verbose output from Clair by setting the `CLAIR_OUTPUT` environment vari ```yaml include: - template: Container-Scanning.gitlab-ci.yml + - template: Container-Scanning.gitlab-ci.yml variables: CLAIR_OUTPUT: High @@ -174,6 +174,7 @@ using environment variables. | `CLAIR_DB_IMAGE_TAG` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | `latest` | | `DOCKERFILE_PATH` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner will look for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | `Dockerfile` | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" | +| `SECURE_LOG_LEVEL` | The log levels available are: `fatal`, `error`, `warn`, `info`, `debug` | `info` | ### Overriding the Container Scanning template @@ -183,7 +184,7 @@ specify any additional keys. For example: ```yaml include: - template: Container-Scanning.gitlab-ci.yml + - template: Container-Scanning.gitlab-ci.yml container_scanning: variables: @@ -195,15 +196,15 @@ GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/REA When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -### Vulnerability whitelisting +### Vulnerability allowlisting -To whitelist specific vulnerabilities, follow these steps: +To allowlist specific vulnerabilities, follow these steps: 1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in [overriding the Container Scanning template](#overriding-the-container-scanning-template). -1. Define the whitelisted vulnerabilities in a YAML file named `clair-whitelist.yml`. This must use - the format described in the [whitelist example file](https://github.com/arminc/clair-scanner/blob/v12/example-whitelist.yaml). -1. Add the `clair-whitelist.yml` file to your project's Git repository. +1. Define the allowlisted vulnerabilities in a YAML file named `vulnerability-allowlist.yml`. This must use + the format described in the [allowlist example file](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/raw/master/testdata/vulnerability-allowlist.yml). +1. Add the `vulnerability-allowlist.yml` file to your project's Git repository. ### Running Container Scanning in an offline environment @@ -282,7 +283,7 @@ stages: build_latest_vulnerabilities: stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind script: - docker pull arminc/clair-db:latest - docker tag arminc/clair-db:latest $CI_REGISTRY/namespace/clair-vulnerabilities-db diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md new file mode 100644 index 00000000000..85da7d85506 --- /dev/null +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -0,0 +1,117 @@ +--- +stage: Secure +group: Fuzz Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Coverage Guided Fuzz Testing **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3226) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha). + +GitLab allows you to add coverage-guided fuzz testing to your pipelines. This helps you discover +bugs and potential security issues that other QA processes may miss. Coverage-guided fuzzing sends +random inputs to an instrumented version of your application in an effort to cause unexpected +behavior, such as a crash. Such behavior indicates a bug that you should address. + +We recommend that you use fuzz testing in addition to the other security scanners in [GitLab Secure](../index.md) +and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +you can run your coverage guided fuzz tests as part your CI/CD workflow. You can take advantage of +Coverage Guided Fuzzing by including the CI job in your existing `.gitlab-ci.yml` file. + +## Supported fuzzing engines and languages + +GitLab supports these languages through the fuzzing engine listed for each. We currently provide a Docker image for apps written in Go, but you can test the other languages below by providing a Docker image with the fuzz engine to run your app. + +| Language | Fuzzing Engine | Example | +|----------|---------------------------------------------------------------------------|---------| +| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | | +| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | | +| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | | + +## Configuration + +To enable fuzzing, you must +[include](../../../ci/yaml/README.md#includetemplate) +the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml) +provided as part of your GitLab installation. + +To do so, add the following to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: Coverage-Fuzzing.gitlab-ci.yml +``` + +The included template makes available the [hidden job](../../../ci/yaml/README.md#hide-jobs) +`.fuzz_base`, which you must [extend](../../../ci/yaml/README.md#extends) for each of your fuzz +targets. Each fuzz target **must** have a separate job. For example, the +[go-fuzzing-example project](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example) +contains one job that extends `.fuzz_base` for its single fuzz target. + +The `my_fuzz_target` job (the separate job for your fuzz target) does the following: + +- Extends `.fuzz_base`. +- Compiles the fuzz target with [go-fuzz](https://github.com/dvyukov/go-fuzz). +- Runs the target with the `gitlab-cov-fuzz` command, which is available to each job that extends + `.fuzz_base`. +- Runs on a fuzz stage that usually comes after a test stage. + +The `gitlab-cov-fuzz` is a command-line tool that runs the instrumented application. It parses and +analyzes the exception information that the fuzzer outputs. It also downloads the [corpus](#glossary) +and crash events from previous pipelines automatically. This helps your fuzz targets build on the progress of +previous fuzzing jobs. The parsed crash events and data are written to +`gl-coverage-fuzzing-report.json`. + +### Artifacts + +Each fuzzing step outputs these artifacts: + +- `gl-coverage-fuzzing-report.json`: This file's format may change in future releases. +- `artifacts.zip`: This file contains two directories: + - `corpus`: Holds all test cases generated by the current and all previous jobs. + - `crashes`: Holds all crash events the current job encountered as well as those not fixed in + previous jobs. + +### Types of Fuzzing Jobs + +There are two types of jobs: + +- Fuzzing: Standard fuzzing session. You can configure a long session through a user defined + timeout. +- Regression: Run the fuzz targets through the accumulated test cases generated by previous fuzzing + sessions plus fixed crashes from previous sessions. This is usually very quick. + +Here's our current suggestion for configuring your fuzz target's timeout: + +- Set `COVERAGE_FUZZING_BRANCH` to the branch where you want to run long-running (async) fuzzing + jobs. This is `master` by default. +- Use regression or short-running fuzzing jobs for other branches or merge requests. + +This suggestion helps find new bugs on the development branch and catch old bugs in merge requests +(like unit tests). + +You can configure this by passing `--regression=false/true` to `gitlab-cov-fuzz` as the [Go example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example/-/blob/master/.gitlab-ci.yml) +shows. Also note that `gitlab-cov-fuzz` is a wrapper, so you can pass those arguments to configure +any option available in the underlying fuzzing engine. + +### Available variables + +| Environment variable | Description | +|---------------------------|--------------------------------------------------------------------| +| `COVERAGE_FUZZING_BRANCH` | The branch for long-running fuzzing jobs. The default is `master`. | + +### Additional Configuration + +The `gitlab-cov-fuzz` command passes all arguments it receives to the underlying fuzzing engine. You +can therefore use all the options available in that fuzzing engine. For more information on these +options, see the underlying fuzzing engine's documentation. + +### Glossary + +- Seed corpus: The set of test cases given as initial input to the fuzz target. This usually speeds + up the fuzz target substantially. This can be either manually created test cases or auto-generated + with the fuzz target itself from previous runs. +- Corpus: The set of meaningful test cases that are generated while the fuzzer is running. Each + meaningful test case produces new coverage in the tested program. It's advised to re-use the + corpus and pass it to subsequent runs. diff --git a/doc/user/application_security/dast/img/dast_all_v13_0.png b/doc/user/application_security/dast/img/dast_all_v13_0.png Binary files differdeleted file mode 100644 index 7b67fc44fae..00000000000 --- a/doc/user/application_security/dast/img/dast_all_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_on_demand_v13_2.png b/doc/user/application_security/dast/img/dast_on_demand_v13_2.png Binary files differnew file mode 100644 index 00000000000..8a733c27be1 --- /dev/null +++ b/doc/user/application_security/dast/img/dast_on_demand_v13_2.png diff --git a/doc/user/application_security/dast/img/dast_v13_2.png b/doc/user/application_security/dast/img/dast_v13_2.png Binary files differnew file mode 100644 index 00000000000..bbf7944eb40 --- /dev/null +++ b/doc/user/application_security/dast/img/dast_v13_2.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 256daae46d7..d68928d858b 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -9,9 +9,9 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4348) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. -NOTE: **4 of the top 6 attacks were application based.** -Download our whitepaper, -["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +NOTE: **Note:** +The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +explains how **4 of the top 6 attacks were application based**. Download it to learn how to protect your organization. Running [static checks](../sast/index.md) on your code is the first step to detect @@ -36,7 +36,7 @@ NOTE: **Note:** This comparison logic uses only the latest pipeline executed for the target branch's base commit. Running the pipeline on any other commit has no effect on the merge request. - + By clicking on one of the detected linked vulnerabilities, you can see the details and the URL(s) affected. @@ -44,10 +44,10 @@ see the details and the URL(s) affected.  [Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_Application_Security_Testing) -uses the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) +uses the popular open source tool [OWASP Zed Attack Proxy](https://www.zaproxy.org/) to perform an analysis on your running web application. -By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) +By default, DAST executes [ZAP Baseline Scan](https://www.zaproxy.org/docs/docker/baseline-scan/) and performs passive scanning only. It won't actively attack your application. However, DAST can be [configured](#full-scan) to also perform an *active scan*: attack your application and produce a more extensive security report. @@ -143,6 +143,22 @@ The only changes to the site should be from the DAST scanner. Be aware that any changes that users, scheduled tasks, database changes, code changes, other pipelines, or other scanners make to the site during a scan could lead to inaccurate results. +### Hide sensitive information + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in GitLab 13.1. + +HTTP request and response headers may contain sensitive information, including cookies and +authorization credentials. By default, the following headers are masked: + +- `Authorization`. +- `Proxy-Authorization`. +- `Set-Cookie` (values only). +- `Cookie` (values only). + +Using the [`DAST_MASK_HTTP_HEADERS` variable](#available-variables), you can list the +headers whose values you want masked. For details on how to mask headers, see +[Customizing the DAST settings](#customizing-the-dast-settings). + ### Authentication It's also possible to authenticate the user before performing the DAST checks. @@ -398,6 +414,10 @@ variables: ### Customizing the DAST settings +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + The DAST settings can be changed through environment variables by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. These variables are documented in [available variables](#available-variables). @@ -410,68 +430,43 @@ include: variables: DAST_WEBSITE: https://example.com - DAST_TARGET_AVAILABILITY_TIMEOUT: 120 + DAST_SPIDER_MINS: 120 ``` Because the template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable will take precedence. -### Overriding the DAST template - -CAUTION: **Deprecation:** -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) -is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. - -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `dast` job after the -template inclusion and specify any additional keys under it. For example: - -```yaml -include: - - template: DAST.gitlab-ci.yml - -dast: - stage: dast # IMPORTANT: don't forget to add this - variables: - DAST_WEBSITE: https://example.com - CI_DEBUG_TRACE: "true" -``` - -As the DAST job belongs to a separate `dast` stage that runs after all -[default stages](../../../ci/yaml/README.md#stages), -don't forget to add `stage: dast` when you override the template job definition. - ### Available variables DAST can be [configured](#customizing-the-dast-settings) using environment variables. -| Environment variable | Required | Description | +| Environment variable | Type | Description | |-----------------------------| -----------|--------------------------------------------------------------------------------| -| `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address from which to download the analyzer. | -| `DAST_WEBSITE` | no| The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. | -| `DAST_API_SPECIFICATION` | no | The API specification to import. `DAST_WEBSITE` must be specified if this is omitted. | -| `DAST_AUTH_URL` | no | The authentication URL of the website to scan. Not supported for API scans. | -| `DAST_USERNAME` | no | The username to authenticate to in the website. | -| `DAST_PASSWORD` | no | The password to authenticate to in the website. | -| `DAST_USERNAME_FIELD` | no | The name of username field at the sign-in HTML form. | -| `DAST_PASSWORD_FIELD` | no | The name of password field at the sign-in HTML form. | -| `DAST_AUTH_EXCLUDE_URLS` | no | The URLs to skip during the authenticated scan; comma-separated, no spaces in between. Not supported for API scans. | -| `DAST_TARGET_AVAILABILITY_TIMEOUT` | no | Time limit in seconds to wait for target availability. Scan is attempted nevertheless if it runs out. Integer. Defaults to `60`. | -| `DAST_FULL_SCAN_ENABLED` | no | Switches the tool to execute [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | -| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | no | Requires [domain validation](#domain-validation) when running DAST full scans. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. Not supported for API scans. | -| `DAST_AUTO_UPDATE_ADDONS` | no | By default the versions of ZAP add-ons are pinned to those provided with the DAST image. Set to `true` to allow ZAP to download the latest versions. | -| `DAST_API_HOST_OVERRIDE` | no | Used to override domains defined in API specification files. | -| `DAST_EXCLUDE_RULES` | no | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from the scan report. Currently, excluded rules will get executed but the alerts from them will be suppressed. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. | -| `DAST_REQUEST_HEADERS` | no | Set to a comma-separated list of request header names and values. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | -| `DAST_DEBUG` | no | Enable debug message output. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | -| `DAST_SPIDER_MINS` | no | The maximum duration of the spider scan in minutes. Set to zero for unlimited. Defaults to one minute, or unlimited when the scan is a full scan. | -| `DAST_HTML_REPORT` | no | The file name of the HTML report written at the end of a scan. | -| `DAST_MARKDOWN_REPORT` | no | The file name of the Markdown report written at the end of a scan. | -| `DAST_XML_REPORT` | no | The file name of the XML report written at the end of a scan. | -| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | no | Include alpha passive and active scan rules. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | -| `DAST_USE_AJAX_SPIDER` | no | Use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | -| `DAST_ZAP_CLI_OPTIONS` | no | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. | -| `DAST_ZAP_LOG_CONFIGURATION` | no | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG` | +| `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. | +| `DAST_WEBSITE` | URL | The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. | +| `DAST_API_SPECIFICATION` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_AUTH_URL` | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` will be submitted with the login form to create an authenticated scan. Not supported for API scans. | +| `DAST_USERNAME` | string | The username to authenticate to in the website. | +| `DAST_PASSWORD` | string | The password to authenticate to in the website. | +| `DAST_USERNAME_FIELD` | string | The name of username field at the sign-in HTML form. | +| `DAST_PASSWORD_FIELD` | string | The name of password field at the sign-in HTML form. | +| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (introduced in GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | +| `DAST_AUTH_EXCLUDE_URLS` | URLs | The URLs to skip during the authenticated scan; comma-separated, no spaces in between. Not supported for API scans. | +| `DAST_FULL_SCAN_ENABLED` | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | +| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | Set to `true` to require [domain validation](#domain-validation) when running DAST full scans. Not supported for API scans. Default: `false` | +| `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | +| `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Example: `example.com:8080` | +| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were supressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_REQUEST_HEADERS` | string | Set to a comma-separated list of request header names and values. Headers will be added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | +| `DAST_DEBUG` | boolean | Enable debug message output. Default: `false` | +| `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. | +| `DAST_HTML_REPORT` | string | The file name of the HTML report written at the end of a scan. | +| `DAST_MARKDOWN_REPORT` | string | The file name of the Markdown report written at the end of a scan. | +| `DAST_XML_REPORT` | string | The file name of the XML report written at the end of a scan. | +| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false` | +| `DAST_USE_AJAX_SPIDER` | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false` | +| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. | +| `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` | ### DAST command-line options @@ -532,19 +527,20 @@ A DAST job has two executing processes: Debug mode of the scripts can be enabled by using the `DAST_DEBUG` environment variable. This can help when troubleshooting the job, and will output statements indicating what percentage of the scan is complete. -For details on using variables, see [Overriding the DAST template](#overriding-the-dast-template). +For details on using variables, see [Overriding the DAST template](#customizing-the-dast-settings). Debug mode of the ZAP server can be enabled using the `DAST_ZAP_LOG_CONFIGURATION` environment variable. The following table outlines examples of values that can be set and the effect that they have on the output that is logged. Multiple values can be specified, separated by semicolons. -| Log configuration value | Effect | -|-------------------------------------------------- | ----------------------------------------------------------------- | -| `log4j.rootLogger=DEBUG` | Enable all debug logging statements. | -| `log4j.logger.org.apache.commons.httpclient=DEBUG` | Log every HTTP request and response made by the ZAP server. | -| `log4j.logger.com.crawljax=DEBUG` | Enable Ajax Crawler debug logging statements. | -| `log4j.logger.org.parosproxy.paros=DEBUG` | Enable ZAP server proxy debug logging statements. | -| `log4j.logger.org.zaproxy.zap=DEBUG` | Enable debug logging statements of the general ZAP server code. | +| Log configuration value | Effect | +|-------------------------------------------------- | ----------------------------------------------------------------- | +| `log4j.rootLogger=DEBUG` | Enable all debug logging statements. | +| `log4j.logger.org.apache.commons.httpclient=DEBUG` | Log every HTTP request and response made by the ZAP server. | +| `log4j.logger.org.zaproxy.zap.spider.SpiderController=DEBUG` | Log URLs found during the spider scan of the target. | +| `log4j.logger.com.crawljax=DEBUG` | Enable Ajax Crawler debug logging statements. | +| `log4j.logger.org.parosproxy.paros=DEBUG` | Enable ZAP server proxy debug logging statements. | +| `log4j.logger.org.zaproxy.zap=DEBUG` | Enable debug logging statements of the general ZAP server code. | ## Running DAST in an offline environment @@ -604,6 +600,44 @@ security reports without requiring internet access. Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. +## On-Demand Scans + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. +> - It's deployed behind a feature flag, disabled by default. +> - It's disabled on GitLab.com. +> - It's able to be enabled or disabled per-project. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans). + +Passive DAST scans may be run on demand against a target website, outside the DevOps lifecycle. These scans will +always be associated with the default or `master` branch of your project and the results can be seen in the project dashboard. + + + +### Enable or disable On-Demand Scans + +On-Demand Scans is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. On-Demand Scans can be enabled or disabled per-project + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:security_on_demand_scans_feature_flag) +# or by project +Feature.enable(:security_on_demand_scans_feature_flag, Project.find(<project id>)) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:security_on_demand_scans_feature_flag) +# or by project +Feature.disable(:security_on_demand_scans_feature_flag, Project.find(<project id>)) +``` + ## Reports The DAST tool outputs a report file in JSON format by default. However, this tool can also generate reports in @@ -683,18 +717,6 @@ Once a vulnerability is found, you can interact with it. Read more on how to For more information about the vulnerabilities database update, check the [maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> - ## Optimizing DAST By default, DAST will download all artifacts defined by previous jobs in the pipeline. If @@ -734,3 +756,15 @@ variables: Here, DAST is being allocated 3072 MB. Change the number after `-Xmx` to the required memory amount. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 474f9339d0b..ca2b212ffc3 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -1,3 +1,10 @@ +--- +type: reference, howto +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Dependency Scanning Analyzers **(ULTIMATE)** Dependency Scanning relies on underlying third party tools that are wrapped into diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png Binary files differdeleted file mode 100644 index 9f3990df957..00000000000 --- a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png Binary files differnew file mode 100644 index 00000000000..28c4eb85b7c --- /dev/null +++ b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 84ec0ec976d..57b4fae3230 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -27,7 +27,7 @@ GitLab checks the Dependency Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. - + The results are sorted by the severity of the vulnerability: @@ -61,7 +61,7 @@ The following languages and dependency managers are supported: | Language (package managers) | Supported files | Scan tool(s) | |----------------------------- | --------------- | ------------ | | Java ([Gradle](https://gradle.org/), [Maven](https://maven.apache.org/)) | `build.gradle`, `build.gradle.kts`, `pom.xml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js) | +| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | | Go ([Golang](https://golang.org/)) | `go.sum` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | PHP ([Composer](https://getcomposer.org/)) | `composer.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Python ([setuptools](https://setuptools.readthedocs.io/en/latest/), [pip](https://pip.pypa.io/en/stable/), [Pipenv](https://pipenv.pypa.io/en/latest/)) | `setup.py`, `requirements.txt`, `requirements.pip`, `requires.txt`, `Pipfile` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | @@ -72,7 +72,7 @@ Plans are underway for supporting the following languages, dependency managers, | Language (package managers) | Supported files | Scan tool(s) | Issue | |----------------------------- | --------------- | ------------ | ----- | -| Python ([Poetry](https://poetry.eustace.io/)) | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/issues/7006) | +| Python ([Poetry](https://python-poetry.org/)) | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | | Python ([Pipenv](https://pipenv.pypa.io/en/latest/)) | `Pipfile.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#11756](https://gitlab.com/gitlab-org/gitlab/-/issues/11756) | ## Contribute your scanner @@ -151,11 +151,11 @@ The following variables allow configuration of global dependency scanning settin | Environment variable | Description | | --------------------------------------- |------------ | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `DS_ANALYZER_IMAGE_PREFIX` | **DEPRECATED:** Use `SECURE_ANALYZERS_PREFIX` instead. | | `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | | `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"` | +| `SECURE_LOG_LEVEL` | Default log level is `info`, you can set it to any of the following strings: `fatal`, `error`, `warn`, `info`, `debug`. | #### Configuring Docker-in-Docker orchestrator @@ -186,6 +186,7 @@ The following variables are used for configuring specific analyzers (used for a | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | | `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12296) in GitLab 12.1)| +| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`. Maven and Gradle will use the Java version specified by this value. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repos). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | @@ -428,14 +429,14 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set Dependency Scanning CI job variables to use local Dependency Scanning analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must replace -`DS_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: +`SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml variables: - DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers" + SECURE_ANALYZERS_PREFIX: "docker-registry.example.com/analyzers" GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" GIT_SSL_NO_VERIFY: "true" ``` diff --git a/doc/user/application_security/img/security_configuration_page_v13_1.png b/doc/user/application_security/img/security_configuration_page_v13_1.png Binary files differdeleted file mode 100644 index 176c64a9e87..00000000000 --- a/doc/user/application_security/img/security_configuration_page_v13_1.png +++ /dev/null diff --git a/doc/user/application_security/img/security_configuration_page_v13_2.png b/doc/user/application_security/img/security_configuration_page_v13_2.png Binary files differnew file mode 100644 index 00000000000..016328948cc --- /dev/null +++ b/doc/user/application_security/img/security_configuration_page_v13_2.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 49580f494a2..3aca4c59423 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -116,6 +116,44 @@ information with several options:  +### View details of a DAST vulnerability + +Vulnerabilities detected by DAST occur in the live web application. Rectification of these types of +vulnerabilities requires specific information. DAST provides the information required to +investigate and rectify the underlying cause. + +To view details of DAST vulnerabilities: + +1. To see all vulnerabilities detected: + + - In a project, go to the project's **{shield}** **Security & Compliance** page. + - Only in a merge request, go the merge request's **Security** tab. + +1. Click on the vulnerability's description. The following details are provided: + + | Field | Description | +|:-----------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Description | Description of the vulnerability. | +| Project | Namespace and project in which the vulnerability was detected. | +| Method | HTTP method used to detect the vulnerability. | +| URL | URL at which the vulnerability was detected. | +| Request Headers | Headers of the request. | +| Response Status | Response status received from the application. | +| Response Headers | Headers of the response received from the application. | +| Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. | +| Identifiers | Identifiers of the vulnerability. | +| Severity | Severity of the vulnerability. | +| Scanner Type | Type of vulnerability report. | +| Links | Links to further details of the detected vulnerability. | +| Solution | Details of a recommended solution to the vulnerability (optional). | + +#### Hide sensitive information in headers + +HTTP request and response headers may contain sensitive information, including cookies and +authorization credentials. By default, content of specific headers are masked in DAST vulnerability +reports. You can specify the list of all headers to be masked. For details, see +[Hide sensitive information](dast/index.md#hide-sensitive-information). + ### Dismissing a vulnerability To dismiss a vulnerability, you must set its status to Dismissed. Follow these steps to do so: @@ -258,14 +296,16 @@ An approval is optional when a security report: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -To enable License Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) -must be created with the case-sensitive name `License-Check`. This approval group must be set -with the number of approvals required greater than zero. +`License-Check` is an approval rule you can enable to allow an individual or group to approve a +merge request that contains a `denied` license. + +You can enable `License-Check` one of two ways: -Once this group is added to your project, the approval rule is enabled for all Merge Requests. To -configure how this rule behaves, you can choose which licenses to `allow` or `deny` in the -[project policies for License Compliance](../compliance/license_compliance/index.md#project-policies-for-license-compliance) -section. +- Create a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) + with the case-sensitive name `License-Check`. +- Create an approval group in the [project policies section for License Compliance](../compliance/license_compliance/index.md#policies). + You must set this approval group's number of approvals required to greater than zero. Once you + enable this group in your project, the approval rule is enabled for all merge requests. Any code changes cause the approvals required to reset. diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 0aa20bf4373..214044ad783 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -32,7 +32,6 @@ SAST supports the following official analyzers: - [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)) - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) -- [`tslint`](https://gitlab.com/gitlab-org/security-products/analyzers/tslint) (TSLint (TypeScript)) The analyzers are published as Docker images that SAST will use to launch dedicated containers for each analysis. @@ -145,24 +144,24 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Analyzers Data -| Property \ Tool | Apex | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | TSLint Security | -| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | :-------------: | -| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | ✓ | -| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | -| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | +| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | +| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | - ✓ => we have that data - ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content diff --git a/doc/user/application_security/sast/img/sast_v13_0.png b/doc/user/application_security/sast/img/sast_v13_0.png Binary files differdeleted file mode 100644 index b4aea6ea466..00000000000 --- a/doc/user/application_security/sast/img/sast_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/sast/img/sast_v13_2.png b/doc/user/application_security/sast/img/sast_v13_2.png Binary files differnew file mode 100644 index 00000000000..5697ed9beb0 --- /dev/null +++ b/doc/user/application_security/sast/img/sast_v13_2.png diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a5497e3d38c..70d4b513cf9 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -9,9 +9,9 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3775) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. -NOTE: **4 of the top 6 attacks were application based.** -Download our whitepaper, -["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +NOTE: **Note:** +The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +explains how **4 of the top 6 attacks were application based**. Download it to learn how to protect your organization. ## Overview @@ -28,7 +28,7 @@ You can take advantage of SAST by doing one of the following: GitLab checks the SAST report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. - + The results are sorted by the priority of the vulnerability: @@ -58,7 +58,8 @@ If you're using the shared Runners on GitLab.com, this is enabled by default. Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker). -CAUTION: **Caution:** Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. +CAUTION: **Caution:** +Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** If you use your own Runners, make sure the Docker version installed @@ -70,31 +71,54 @@ The following table shows which languages, package managers and frameworks are s | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| -| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | -| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | -| Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | -| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | -| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | -| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | -| Go | [Gosec](https://github.com/securego/gosec) | 10.7 | -| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | -| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | +| Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | +| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | +| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | +| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | +| Go | [Gosec](https://github.com/securego/gosec) | 10.7 | +| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | +| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | | Java ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT) | -| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | +| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8, moved to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2 | | Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | | Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | | React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 | -| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | +| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3, moved to [GitLab Core](https://about.gitlab.com/pricing/) in 13.1 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | -| TypeScript | [`tslint-config-security`](https://github.com/webschik/tslint-config-security/) | 11.9 | +| TypeScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.9, merged with ESLint in 13.2 | NOTE: **Note:** The Java analyzers can also be used for variants like the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html), [Grails](https://grails.org/) and the [Maven wrapper](https://github.com/takari/maven-wrapper). +### Making SAST analyzers available to all GitLab tiers + +All open source (OSS) analyzers are in the process of being reviewed and potentially moved to the GitLab Core tier. Progress can be +tracked in the corresponding +[epic](https://gitlab.com/groups/gitlab-org/-/epics/2098). + +Please note that support for [Docker-in-Docker](#enabling-docker-in-docker) +will not be extended to the GitLab Core tier. + +#### Summary of features per tier + +Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), +as shown in the following table: + +| Capability | In Core | In Ultimate | +|:--------------------------------------------------------------------------|:--------------------|:-------------------| +| [Configure SAST Scanners](#configuration) | **{check-circle}** | **{check-circle}** | +| [Customize SAST Settings](#customizing-the-sast-settings) | **{check-circle}** | **{check-circle}** | +| View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | +| [Presentation of JSON Report in Merge Request](#overview) | **{dotted-circle}** | **{check-circle}** | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) | **{dotted-circle}** | **{check-circle}** | +| [Access to Security Dashboard](#security-dashboard) | **{dotted-circle}** | **{check-circle}** | + ## Contribute your scanner The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate other security scanners into GitLab. @@ -222,7 +246,7 @@ a `before_script` execution to prepare your scan job. To pass your project's dependencies as artifacts, the dependencies must be included in the project's working directory and specified using the `artifacts:path` configuration. -If all dependencies are present, the `-compile=false` flag can be provided to the +If all dependencies are present, the `COMPILE=false` variable can be provided to the analyzer and compilation will be skipped: ```yaml @@ -247,10 +271,9 @@ build: spotbugs-sast: dependencies: - build - script: - - /analyzer run -compile=false variables: MAVEN_REPO_PATH: ./.m2/repository + COMPILE: false artifacts: reports: sast: gl-sast-report.json @@ -266,6 +289,16 @@ See [Analyzer settings](#analyzer-settings) for the complete list of available o SAST can be [configured](#customizing-the-sast-settings) using environment variables. +#### Logging Level + +You can control the verbosity of logs by setting the `SECURE_LOG_LEVEL` env var. The default is set to `info`, you can set it to any of the following levels: + +- `fatal` +- `error` +- `warn` +- `info` +- `debug` + #### Custom Certificate Authority To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle @@ -278,7 +311,6 @@ The following are Docker image-related variables. | Environment variable | Description | |------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `SAST_ANALYZER_IMAGE_PREFIX` | **DEPRECATED**: Use `SECURE_ANALYZERS_PREFIX` instead. | | `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | @@ -287,17 +319,18 @@ The following are Docker image-related variables. Some analyzers make it possible to filter out vulnerabilities under a given threshold. -| Environment variable | Default value | Description | -|-------------------------|---------------|-------------| +| Environment variable | Default value | Description | +|-------------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories will also match patterns. | -| `SAST_BANDIT_EXCLUDED_PATHS` | - | comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*'` | -| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | -| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | -| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | -| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | -| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | -| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | -| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +| `SAST_BANDIT_EXCLUDED_PATHS` | | Comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*, */venv/*'` | +| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | +| `SAST_DISABLE_BABEL` | `false` | Disable Babel processing for the NodeJsScan scanner. Set to `true` to disable Babel processing. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33065) in GitLab 13.2. | +| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | +| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | +| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | +| `SAST_GITLEAKS_COMMIT_FROM` | | The commit a Gitleaks scan starts at. | +| `SAST_GITLEAKS_COMMIT_TO` | | The commit a Gitleaks scan ends at. | +| `SAST_GITLEAKS_HISTORIC_SCAN` | `false` | Flag to enable a historic Gitleaks scan. | #### Docker-in-Docker orchestrator @@ -315,11 +348,12 @@ The following variables configure the Docker-in-Docker orchestrator, and therefo Some analyzers can be customized with environment variables. -| Environment variable | Analyzer | Description | -|-----------------------------|----------|-------------| +| Environment variable | Analyzer | Description | +|---------------------------------------|----------------------|-------------| | `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | | `KUBESEC_HELM_CHARTS_PATH` | Kubesec | Optional path to Helm charts that `helm` will use to generate a Kubernetes manifest that `kubesec` will scan. If dependencies are defined, `helm dependency build` should be ran in a `before_script` to fetch the necessary dependencies. | | `KUBESEC_HELM_OPTIONS` | Kubesec | Additional arguments for the `helm` executable. | +| `COMPILE` | SpotBugs | Set to `false` to disable project compilation and dependency fetching. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195252) in GitLab 13.1. | | `ANT_HOME` | SpotBugs | The `ANT_HOME` environment variable. | | `ANT_PATH` | SpotBugs | Path to the `ant` executable. | | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | @@ -333,6 +367,7 @@ Some analyzers can be customized with environment variables. | `FAIL_NEVER` | SpotBugs | Set to `1` to ignore compilation failure. | | `SAST_GOSEC_CONFIG` | Gosec | Path to configuration for Gosec (optional). | | `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | +| `SEARCH_MAX_DEPTH` | any | Maximum number of directories traversed when searching for source code files. Default: `4`. | #### Custom environment variables @@ -494,7 +529,6 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/secrets:2 registry.gitlab.com/gitlab-org/security-products/analyzers/security-code-scan:2 registry.gitlab.com/gitlab-org/security-products/analyzers/sobelow:2 registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/tslint:2 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -509,7 +543,7 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set SAST CI job variables to use local SAST analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must replace -`SAST_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: +`SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: ```yaml include: diff --git a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png Binary files differdeleted file mode 100644 index 17893610f10..00000000000 --- a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png +++ /dev/null diff --git a/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png b/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png Binary files differnew file mode 100644 index 00000000000..4aa7dd83c8d --- /dev/null +++ b/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 85933c31a34..ea635212c5d 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -25,7 +25,7 @@ GitLab displays identified secrets as part of the SAST reports visibly in a few - Pipelines' **Security** tab - Report in the merge request widget - + ## Use cases @@ -39,7 +39,8 @@ To run Secret Detection jobs, by default, you need GitLab Runner with the [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. If you're using the shared Runners on GitLab.com, this is enabled by default. -CAUTION: **Caution:** Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. +CAUTION: **Caution:** +Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** If you use your own Runners, make sure the Docker version installed @@ -118,15 +119,15 @@ declare a job with the same name as the SAST job to override. Place this new job inclusion and specify any additional keys under it. In the following example, we include the Secret Detection template and at the same time we -override the `secret-scan` job with the `SECRET_DETECTION_HISTORIC_SCAN` variable to `true`: +override the `secret_detection` job with the `SECRET_DETECTION_HISTORIC_SCAN` variable to `true`: ```yaml include: - template: Secret-Detection.gitlab-ci.yml -secrets-scan: +secret_detection: variables: - SECRET_DETECTION_HISTORIC_SCAN: true + SECRET_DETECTION_HISTORIC_SCAN: "true" ``` Because the template is [evaluated before](../../../ci/yaml/README.md#include) @@ -146,6 +147,16 @@ Secret Detection can be customized by defining available variables: | `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +### Logging Level + +You can control the verbosity of logs by setting the `SECURE_LOG_LEVEL` env var. The default is set to `info`, you can set it to any of the following levels: + +- `fatal` +- `error` +- `warn` +- `info` +- `debug` + ## Full History Secret Scan GitLab 12.11 introduced support for scanning the full history of a repository. This new functionality diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png Binary files differindex 0dfe7b637cd..d98fb71ae37 100644 --- a/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png Binary files differdeleted file mode 100644 index 4c7b5cc724f..00000000000 --- a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png Binary files differnew file mode 100644 index 00000000000..d6cfc2de980 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png Binary files differdeleted file mode 100644 index a500f186c2b..00000000000 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png Binary files differnew file mode 100644 index 00000000000..75b5ad1d885 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png Binary files differdeleted file mode 100644 index 670c90d10a3..00000000000 --- a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png Binary files differnew file mode 100644 index 00000000000..591a08f4d7a --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png Binary files differnew file mode 100644 index 00000000000..7cab7b0a61f --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png diff --git a/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png b/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png Binary files differnew file mode 100644 index 00000000000..9cf95b197fe --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png diff --git a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png Binary files differnew file mode 100644 index 00000000000..2b792727a99 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 60798b9c921..9a13d143d1f 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Secure +group: Threat Insights +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Security Dashboard **(ULTIMATE)** @@ -9,7 +12,7 @@ vulnerabilities in your groups, projects and pipelines. You can also drill down into a vulnerability and get extra information, see which project it comes from, the file it's in, and various metadata to help you analyze -the risk. You can also action these vulnerabilities by creating an issue for them, +the risk. You can also take actions on vulnerabilities by creating an issue for them, or by dismissing them. To benefit from the Security Dashboard you must first configure one of the @@ -42,7 +45,7 @@ At the pipeline level, the Security section displays the vulnerabilities present Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security findings. - + NOTE: **Note:** A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. @@ -51,56 +54,52 @@ A pipeline consists of multiple jobs, including SAST and DAST scanning. If any j > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6165) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.1. -At the project level, the Security Dashboard displays the latest security reports for your project. -Use it to find and fix vulnerabilities. +At the project level, the Security Dashboard displays the vulnerabilities merged into your project's +[default branch](../../project/repository/branches/index.md#default-branch). Access it by navigating +to **Security & Compliance > Security Dashboard**. - +The Security Dashboard first displays the total number of vulnerabilities by severity (for example, +Critical, High, Medium, Low). Below this, a table displays each vulnerability's status, severity, +and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) +page to view more information about that vulnerability. -### Export vulnerabilities +You can filter the vulnerabilities by: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +- Status +- Severity +- Report type -You can export all your project's vulnerabilities as CSV by clicking on the export button located at top right of the Project Security Dashboard. This will initiate the process, and once complete, the CSV report will be downloaded. The report will contain all vulnerabilities in the project as filters won't apply. +You can also dismiss vulnerabilities in the table: -NOTE: **Note:** -It may take several minutes for the download to start if your project consists -of thousands of vulnerabilities. Do not close the page until the download finishes. +1. Select the checkbox for each vulnerability you want to dismiss. +1. In the menu that appears, select the reason for dismissal and click **Dismiss Selected**. - + ## Group Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6709) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5. -The group Security Dashboard gives an overview of the vulnerabilities of all the -projects in a group and its subgroups. +The group Security Dashboard gives an overview of the vulnerabilities in the default branches of the +projects in a group and its subgroups. Access it by navigating to **Security > Security Dashboard** +for your group. -First, navigate to the Security Dashboard found under your group's -**Security** tab. +NOTE: **Note:** +The Security Dashboard only shows projects with [security reports](#supported-reports) enabled in a +group. + + -Once you're on the dashboard, at the top you should see a series of filters for: +You can filter which vulnerabilities the Security Dashboard displays by: - Status - Severity - Report type +- Project -NOTE: **Note:** -The dashboard only shows projects with [security reports](#supported-reports) enabled in a group. - - - -Selecting one or more filters will filter the results in this page. - -The main section is a list of all the vulnerabilities in the group, sorted by severity. -In that list, you can see the severity of the vulnerability, its name, its -confidence (likelihood of the vulnerability to be a positive one), and the project -it's from. - -If you hover over a row, the following actions appear: - -- More info -- Create issue -- Dismiss vulnerability +A table lists the vulnerabilities, sorted by severity. The table shows each vulnerability's status, +severity, and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) +page to view more information about that vulnerability. Next to the list is a timeline chart that shows how many open vulnerabilities your projects had at various points in time. You can filter among 30, 60, and @@ -120,28 +119,14 @@ vulnerabilities are not included either. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). -### Export vulnerabilities - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213013) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. - -You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** button -located at the top right of the **Group Security Dashboard**. After the report builds, the CSV -report downloads to your local machine. The report contains all vulnerabilities for the projects -defined in the **Group Security Dashboard**, as filters don't apply to the export function. - -NOTE: **Note:** -It may take several minutes for the download to start if your project contains thousands of -vulnerabilities. Don't close the page until the download finishes. - - - ## Instance Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6953) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.8. -At the instance level, the Security Dashboard displays the vulnerabilities -present in all of the projects that you have added to it. It includes all -of the features of the [group security dashboard](#group-security-dashboard). +At the instance level, the Security Dashboard displays the vulnerabilities present in the default +branches of all the projects you configure to display on the dashboard. It includes all the +[group Security Dashboard's](#group-security-dashboard) +features. You can access the Instance Security Dashboard from the menu bar at the top of the page. Under **More**, select **Security**. @@ -156,27 +141,25 @@ To add projects to the dashboard: 1. Search for and add one or more projects using the **Search your projects** field. 1. Click the **Add projects** button. -Once added, the dashboard will display the vulnerabilities found in your chosen -projects. +Once added, the Security Dashboard displays the vulnerabilities found in your chosen projects' +default branches. - + -### Export vulnerabilities +## Export vulnerabilities -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. -You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** -button located at top right of the **Instance Security Dashboard**. After the report +You can export all your vulnerabilities in CSV format by clicking the **{upload}** **Export** +button located at top right of the **Security Dashboard**. After the report is built, the CSV report downloads to your local machine. The report contains all -vulnerabilities for the projects defined in the **Instance Security Dashboard**, +vulnerabilities for the projects defined in the **Security Dashboard**, as filters don't apply to the export function. NOTE: **Note:** It may take several minutes for the download to start if your project contains thousands of vulnerabilities. Do not close the page until the download finishes. - - ## Keeping the dashboards up to date The Security Dashboard displays information from the results of the most recent @@ -194,12 +177,34 @@ Dashboard regardless of how often the default branch is updated. That way, reports are created even if no code change happens. +CAUTION: **Warning:** +Running Dependency Scanning from a scheduled pipeline might result in false negatives if your +project doesn't have a lock file and isn't configured for Continuous Delivery. A lock file is a file +that lists all transient dependencies and keeps track of their exact versions. The false negative +can occur because the dependency version resolved during the scan might differ from the ones +resolved when your project was built and released, in a previous pipeline. Java projects can't have +lock files. Python projects can have lock files, but GitLab Secure tools don't support them. + ## Security scans using Auto DevOps When using [Auto DevOps](../../../topics/autodevops/index.md), use [special environment variables](../../../topics/autodevops/customize.md#environment-variables) to configure daily security scans. +## Vulnerability list + +Each dashboard's vulnerability list contains vulnerabilities from the latest scans that were merged +into the default branch. +Click any vulnerability in the table to see more information on that vulnerability. To create an +issue associated with the vulnerability, click the **Create Issue** button. + + + +Once you create the issue, the vulnerability list contains a link to the issue and an icon whose +color indicates the issue's status (green for open issues, blue for closed issues). + + + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index 434048896fe..a6738677454 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -58,12 +58,15 @@ prerequisites: If you're using custom Helm values for Cilium, you must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Hubble values](../../clusters/applications.md#install-cilium-using-gitlab-cicd): +your [Cilium values](../../clusters/applications.md#install-cilium-using-gitlab-cicd): ```yaml -metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' +global: + hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' ``` The **Container Network Policy** section displays the following information diff --git a/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v12_10.png b/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v12_10.png Binary files differdeleted file mode 100644 index 0fdb8d1e201..00000000000 --- a/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v12_10.png +++ /dev/null diff --git a/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v13_1.png b/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v13_1.png Binary files differnew file mode 100644 index 00000000000..e0e0fdb6f6e --- /dev/null +++ b/doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v13_1.png diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index b3128e49980..d5cce6434d8 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -1,7 +1,7 @@ --- type: reference, howto stage: Secure -group: Vulnerability Research +group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -9,10 +9,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. -Each security vulnerability in the [Vulnerability List](../dependency_list/index.md) has its own standalone +Each security vulnerability in the [Security Dashboard](../security_dashboard/index.md#project-security-dashboard) has its own standalone page. - + On the standalone vulnerability page, you can interact with the vulnerability in several different ways: @@ -30,7 +30,7 @@ several different ways: You can switch the status of a vulnerability using the **Status** dropdown to one of the following values: -| State | Description | +| Status | Description | |-----------|-------------------------------------------------------------------| | Detected | The default state for a newly discovered vulnerability | | Confirmed | A user has seen this vulnerability and confirmed it to be real | diff --git a/doc/user/asciidoc.md b/doc/user/asciidoc.md index 8834deb8d50..512a98d567b 100644 --- a/doc/user/asciidoc.md +++ b/doc/user/asciidoc.md @@ -10,14 +10,14 @@ You can find the full documentation for the AsciiDoc syntax at <https://asciidoc ### Paragraphs -```asciidoc +```plaintext A normal paragraph. Line breaks are not preserved. ``` Line comments, which are lines that start with `//`, are skipped: -```asciidoc +```plaintext // this is a comment ``` @@ -25,7 +25,7 @@ A blank line separates paragraphs. A paragraph with the `[%hardbreaks]` option will preserve line breaks: -```asciidoc +```plaintext [%hardbreaks] This paragraph carries the `hardbreaks` option. Notice how line breaks are now preserved. @@ -35,7 +35,7 @@ An indented (literal) paragraph disables text formatting, preserves spaces and line breaks, and is displayed in a monospaced font: -```asciidoc +```plaintext This literal paragraph is indented with one space. As a consequence, *text formatting*, spaces, and lines breaks will be preserved. @@ -43,7 +43,7 @@ monospaced font: An admonition paragraph grabs the reader's attention: -```asciidoc +```plaintext NOTE: This is a brief reference, please read the full documentation at https://asciidoctor.org/docs/. TIP: Lists can be indented. Leading whitespace is not significant. @@ -53,7 +53,7 @@ TIP: Lists can be indented. Leading whitespace is not significant. **Constrained (applied at word boundaries)** -```asciidoc +```plaintext *strong importance* (aka bold) _stress emphasis_ (aka italic) `monospaced` (aka typewriter text) @@ -64,7 +64,7 @@ _stress emphasis_ (aka italic) **Unconstrained (applied anywhere)** -```asciidoc +```plaintext **C**reate+**R**ead+**U**pdate+**D**elete fan__freakin__tastic ``mono``culture @@ -72,7 +72,7 @@ fan__freakin__tastic **Replacements** -```asciidoc +```plaintext A long time ago in a galaxy far, far away... (C) 1976 Arty Artisan I believe I shall--no, actually I won't. @@ -80,7 +80,7 @@ I believe I shall--no, actually I won't. **Macros** -```asciidoc +```plaintext // where c=specialchars, q=quotes, a=attributes, r=replacements, m=macros, p=post_replacements, etc. The European icon:flag[role=blue] is blue & contains pass:[************] arranged in a icon:circle-o[role=yellow]. The pass:c[->] operator is often referred to as the stabby lambda. @@ -93,12 +93,12 @@ stem:[sqrt(4) = 2] **User-defined attributes** -```asciidoc +```plaintext // define attributes in the document header :name: value ``` -```asciidoc +```plaintext :url-gem: https://rubygems.org/gems/asciidoctor You can download and install Asciidoctor {asciidoctor-version} from {url-gem}. @@ -117,7 +117,7 @@ GitLab sets the following environment attributes: ### Links -```asciidoc +```plaintext https://example.org/page[A webpage] link:../path/to/file.txt[A local file] xref:document.adoc[A sibling document] @@ -126,7 +126,7 @@ mailto:hello@example.org[Email to say hello!] ### Anchors -```asciidoc +```plaintext [[idname,reference text]] // or written using normal block attributes as `[#idname,reftext=reference text]` A paragraph (or any block) with an anchor (aka ID) and reftext. @@ -142,7 +142,7 @@ This paragraph has a footnote.footnote:[This is the text of the footnote.] #### Unordered -```asciidoc +```plaintext * level 1 ** level 2 *** level 3 @@ -161,7 +161,7 @@ Attach a block or paragraph to a list item using a list continuation (which you #### Ordered -```asciidoc +```plaintext . Step 1 . Step 2 .. Step 2a @@ -177,14 +177,14 @@ Attach a block or paragraph to a list item using a list continuation (which you #### Checklist -```asciidoc +```plaintext * [x] checked * [ ] not checked ``` #### Callout -```asciidoc +```plaintext // enable callout bubbles by adding `:icons: font` to the document header [,ruby] ---- @@ -195,7 +195,7 @@ puts 'Hello, World!' # <1> #### Description -```asciidoc +```plaintext first term:: description of first term second term:: description of second term @@ -205,7 +205,7 @@ description of second term #### Header -```asciidoc +```plaintext = Document Title Author Name <author@example.org> v1.0, 2019-01-01 @@ -213,7 +213,7 @@ v1.0, 2019-01-01 #### Sections -```asciidoc +```plaintext = Document Title (Level 0) == Level 1 === Level 2 @@ -225,7 +225,7 @@ v1.0, 2019-01-01 #### Includes -```asciidoc +```plaintext include::basics.adoc[] // define -a allow-uri-read to allow content to be read from URI @@ -239,13 +239,13 @@ included, a number that is inclusive of transitive dependencies. ### Blocks -```asciidoc +```plaintext -- open - a general-purpose content wrapper; useful for enclosing content to attach to a list item -- ``` -```asciidoc +```plaintext // recognized types include CAUTION, IMPORTANT, NOTE, TIP, and WARNING // enable admonition icons by setting `:icons: font` in the document header [NOTE] @@ -254,13 +254,13 @@ admonition - a notice for the reader, ranging in severity from a tip to an alert ==== ``` -```asciidoc +```plaintext ==== example - a demonstration of the concept being documented ==== ``` -```asciidoc +```plaintext .Toggle Me [%collapsible] ==== @@ -268,58 +268,58 @@ collapsible - these details are revealed by clicking the title ==== ``` -```asciidoc +```plaintext **** sidebar - auxiliary content that can be read independently of the main content **** ``` -```asciidoc +```plaintext .... literal - an exhibit that features program output .... ``` -```asciidoc +```plaintext ---- listing - an exhibit that features program input, source code, or the contents of a file ---- ``` -```asciidoc +```plaintext [,language] ---- source - a listing that is embellished with (colorized) syntax highlighting ---- ``` -````asciidoc +````plaintext \```language fenced code - a shorthand syntax for the source block \``` ```` -```asciidoc +```plaintext [,attribution,citetitle] ____ quote - a quotation or excerpt; attribution with title of source are optional ____ ``` -```asciidoc +```plaintext [verse,attribution,citetitle] ____ verse - a literary excerpt, often a poem; attribution with title of source are optional ____ ``` -```asciidoc +```plaintext ++++ pass - content passed directly to the output document; often raw HTML ++++ ``` -```asciidoc +```plaintext // activate stem support by adding `:stem:` to the document header [stem] ++++ @@ -327,7 +327,7 @@ x = y^2 ++++ ``` -```asciidoc +```plaintext //// comment - content which is not included in the output document //// @@ -335,7 +335,7 @@ comment - content which is not included in the output document ### Tables -```asciidoc +```plaintext .Table Attributes [cols=>1h;2d,width=50%,frame=topbot] |=== @@ -366,7 +366,7 @@ comment - content which is not included in the output document ### Multimedia -```asciidoc +```plaintext image::screenshot.png[block image,800,450] Press image:reload.svg[reload,16,opts=interactive] to reload the page. @@ -380,12 +380,12 @@ video::300817511[vimeo] ### Breaks -```asciidoc +```plaintext // thematic break (aka horizontal rule) --- ``` -```asciidoc +```plaintext // page break <<< ``` diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 86624d12bcf..507ba25850d 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -32,8 +32,6 @@ To see a list of available applications to install. For a: - [Group-level cluster](../group/clusters/index.md), navigate to your group's **{cloud-gear}** **Kubernetes** page. -Install Helm first as it's used to install other applications. - NOTE: **Note:** As of GitLab 11.6, Helm will be upgraded to the latest version supported by GitLab before installing any of the applications. @@ -71,17 +69,47 @@ can lead to confusion during deployments. > - Introduced in GitLab 10.2 for project-level clusters. > - Introduced in GitLab 11.6 for group-level clusters. +> - A local Tiller option was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209736) in GitLab 13.2 behind a feature flag, enabled by default. +> - The feature flag for local Tiller is enabled on GitLab.com. [Helm](https://helm.sh/docs/) is a package manager for Kubernetes and is -required to install all the other applications. It is installed in its -own pod inside the cluster which can run the `helm` CLI in a safe -environment. +used to install the GitLab-managed apps. GitLab runs each `helm` command +in a pod within the `gitlab-managed-apps` namespace inside the cluster. + +As of GitLab 13.2, the integration uses a local +[Tiller](https://v2.helm.sh/docs/glossary/#tiller) by default. When using a +local Tiller, the Helm application does not need to be installed and will not +be shown in the list of applications. NOTE: **Note:** -Installing Helm as a GitLab-managed App behind a proxy is not supported, -but a [workaround](../../topics/autodevops/index.md#installing-helm-behind-a-proxy) +GitLab's Helm integration does not support installing applications behind a proxy, +but a [workaround](../../topics/autodevops/index.md#install-applications-behind-a-proxy) is available. +### Enable or disable local Tiller **(CORE ONLY)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209736) in GitLab 13.2 +> - The option to disable local Tiller is [planned for removal](https://gitlab.com/gitlab-org/gitlab/-/issues/209736) in GitLab 13.3 + +Local Tiller is under development, but is ready for production use. It is +deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it for your instance. + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:managed_apps_local_tiller) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:managed_apps_local_tiller) +``` + ### cert-manager > Introduced in GitLab 11.6 for project- and group-level clusters. @@ -609,6 +637,7 @@ Supported applications: - [Sentry](#install-sentry-using-gitlab-cicd) - [GitLab Runner](#install-gitlab-runner-using-gitlab-cicd) - [Cilium](#install-cilium-using-gitlab-cicd) +- [Falco](#install-falco-using-gitlab-cicd) - [Vault](#install-vault-using-gitlab-cicd) - [JupyterHub](#install-jupyterhub-using-gitlab-cicd) - [Elastic Stack](#install-elastic-stack-using-gitlab-cicd) @@ -634,6 +663,11 @@ To install applications using GitLab CI/CD: - template: Managed-Cluster-Applications.gitlab-ci.yml ``` + NOTE: **Note:** + The job provided by this template connects to the cluster using tools provided + in a custom Docker image. It requires that you have a runner registered with the Docker, + Kubernetes, or Docker Machine executor. + 1. Add a `.gitlab/managed-apps/config.yaml` file to define which applications you would like to install. Define the `installed` key as `true` to install the application and `false` to uninstall the @@ -683,6 +717,10 @@ management project. Refer to the [chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) for the available configuration options. +NOTE: **Note:** +Support for installing the Ingress managed application is provided by the GitLab Configure group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). + ### Install cert-manager using GitLab CI/CD cert-manager is installed using GitLab CI/CD by defining configuration in @@ -720,6 +758,10 @@ management project. Refer to the [chart](https://hub.helm.sh/charts/jetstack/cert-manager) for the available configuration options. +NOTE: **Note:** +Support for installing the Cert Manager managed application is provided by the GitLab Configure group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). + ### Install Sentry using GitLab CI/CD NOTE: **Note:** @@ -781,6 +823,10 @@ postgresql: postgresqlPassword: example-postgresql-password ``` +NOTE: **Note:** +Support for installing the Sentry managed application is provided by the GitLab Health group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Health group](https://about.gitlab.com/handbook/product/product-categories/#health-group). + ### Install PostHog using GitLab CI/CD [PostHog](https://www.posthog.com) 🦔 is a developer-friendly, open-source product analytics platform. @@ -852,6 +898,10 @@ project. Refer to the [Configuration section of the Prometheus chart's README](https://github.com/helm/charts/tree/master/stable/prometheus#configuration) for the available configuration options. +NOTE: **Note:** +Support for installing the Prometheus managed application is provided by the GitLab APM group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). + ### Install GitLab Runner using GitLab CI/CD GitLab Runner is installed using GitLab CI/CD by defining configuration in @@ -883,6 +933,10 @@ management project. Refer to the [chart](https://gitlab.com/gitlab-org/charts/gitlab-runner) for the available configuration options. +NOTE: **Note:** +Support for installing the Runner managed application is provided by the GitLab Runner group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Runner group](https://about.gitlab.com/handbook/product/product-categories/#runner-group). + ### Install Cilium using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/22) in GitLab 12.8. @@ -909,8 +963,8 @@ a corresponding cluster type. The default value is blank. You can check the recommended variables for each cluster type in the official documentation: -- [Google GKE](https://cilium.readthedocs.io/en/stable/gettingstarted/k8s-install-gke/#prepare-deploy-cilium) -- [AWS EKS](https://cilium.readthedocs.io/en/stable/gettingstarted/k8s-install-eks/#prepare-deploy-cilium) +- [Google GKE](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/#deploy-cilium) +- [AWS EKS](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-eks/#deploy-cilium) You can customize Cilium's Helm variables by defining the `.gitlab/managed-apps/cilium/values.yaml` file in your cluster @@ -920,9 +974,9 @@ for the available configuration options. CAUTION: **Caution:** Installation and removal of the Cilium requires a **manual** -[restart](https://cilium.readthedocs.io/en/stable/gettingstarted/k8s-install-gke/#restart-remaining-pods) +[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/#restart-unmanaged-pods) of all affected pods in all namespaces to ensure that they are -[managed](https://cilium.readthedocs.io/en/stable/troubleshooting/#ensure-pod-is-managed-by-cilium) +[managed](https://docs.cilium.io/en/stable/troubleshooting/#ensure-pod-is-managed-by-cilium) by the correct networking plugin. NOTE: **Note:** @@ -930,23 +984,20 @@ Major upgrades might require additional setup steps, please consult the official [upgrade guide](https://docs.cilium.io/en/stable/install/upgrade/) for more information. -By default, Cilium will drop all disallowed packets upon policy -deployment. The audit mode is scheduled for release in -[Cilium 1.8](https://github.com/cilium/cilium/pull/9970). In the audit -mode, disallowed packets will not be dropped, and audit -notifications will be generated instead. GitLab provides alternative Docker -images for Cilium with the audit patch included. You can switch to the -custom build and enable the audit mode by adding the following to +By default, Cilium's [audit +mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/?highlight=policy-audit#enable-policy-audit-mode) +is enabled. In audit mode, Cilium doesn't drop disallowed packets. You +can use `policy-verdict` log to observe policy-related decisions. You +can disable audit mode by adding the following to `.gitlab/managed-apps/cilium/values.yaml`: ```yaml -global: - registry: registry.gitlab.com/gitlab-org/defend/cilium - policyAuditMode: true +config: + policyAuditMode: false agent: monitor: - eventTypes: ["drop", "audit"] + eventTypes: ["drop"] ``` The Cilium monitor log for traffic is logged out by the @@ -968,24 +1019,121 @@ The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default and it's set to collect per namespace flow metrics. This metrics are accessible on the [Threat Monitoring](../application_security/threat_monitoring/index.md) dashboard. You can disable Hubble by adding the following to -`.gitlab/managed-apps/config.yaml`: +`.gitlab/managed-apps/cilium/values.yaml`: ```yaml -cilium: - installed: true +global: hubble: - installed: false + enabled: false ``` You can also adjust Helm values for Hubble via -`.gitlab/managed-apps/cilium/hubble-values.yaml`: +`.gitlab/managed-apps/cilium/values.yaml`: ```yaml -metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' +global: + hubble: + enabled: true + metrics: + enabled: + - 'flow:sourceContext=namespace;destinationContext=namespace' +``` + +NOTE: **Note:** +Support for installing the Cilium managed application is provided by the GitLab Container Security group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). + +### Install Falco using GitLab CI/CD + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/91) in GitLab 13.1. + +GitLab Container Host Security Monitoring uses [Falco](https://falco.org/) +as a runtime security tool that listens to the Linux kernel using eBPF. Falco parses system calls +and asserts the stream against a configurable rules engine in real-time. For more information, see +[Falco's Documentation](https://falco.org/docs/). + +You can enable Falco in the +`.gitlab/managed-apps/config.yaml` file: + +```yaml +falco: + installed: true +``` + +You can customize Falco's Helm variables by defining the +`.gitlab/managed-apps/falco/values.yaml` file in your cluster +management project. Refer to the +[Falco chart](https://github.com/falcosecurity/charts/tree/master/falco) +for the available configuration options. + +CAUTION: **Caution:** +By default eBPF support is enabled and Falco will use an [eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) to pass system calls to userspace. +If your cluster doesn't support this, you can configure it to use Falco kernel module instead by adding the following to `.gitlab/managed-apps/falco/values.yaml`: + +```yaml +ebpf: + enabled: false ``` +In rare cases where automatic probe installation on your cluster isn't possible and the kernel/probe +isn't precompiled, you may need to manually prepare the kernel module or eBPF probe with +[driverkit](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) +and install it on each cluster node. + +By default, Falco is deployed with a limited set of rules. To add more rules, add the following to +`.gitlab/managed-apps/falco/values.yaml` (you can get examples from +[Cloud Native Security Hub](https://securityhub.dev/)): + +```yaml +customRules: + file-integrity.yaml: |- + - rule: Detect New File + desc: detect new file created + condition: > + evt.type = chmod or evt.type = fchmod + output: > + File below a known directory opened for writing (user=%user.name + command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) + priority: ERROR + tags: [filesystem] + - rule: Detect New Directory + desc: detect new directory created + condition: > + mkdir + output: > + File below a known directory opened for writing (user=%user.name + command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) + priority: ERROR + tags: [filesystem] +``` + +By default, Falco only outputs security events to logs as JSON objects. To set it to output to an +[external API](https://falco.org/docs/alerts/#https-output-send-alerts-to-an-https-end-point) +or [application](https://falco.org/docs/alerts/#program-output), +add the following to `.gitlab/managed-apps/falco/values.yaml`: + +```yaml +falco: + programOutput: + enabled: true + keepAlive: false + program: mail -s "Falco Notification" someone@example.com + + httpOutput: + enabled: true + url: http://some.url +``` + +You can check these logs with the following command: + +```shell +kubectl logs -l app=falco -n gitlab-managed-apps +``` + +NOTE: **Note:** +Support for installing the Falco managed application is provided by the GitLab Container Security group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). + ### Install Vault using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9982) in GitLab 12.9. @@ -1035,7 +1183,7 @@ below are examples and should be replaced with settings specific to your environ ui: enabled: true server: - # Disable the built in data storage volume as it's not safe for Hight Availablity mode + # Disable the built in data storage volume as it's not safe for Hight Availability mode dataStorage: enabled: false # Enable High Availability Mode @@ -1075,6 +1223,10 @@ kubectl -n gitlab-managed-apps exec -it vault-0 sh This should give you your unseal keys and initial root token. Make sure to note these down and keep these safe as you will need them to unseal the Vault throughout its lifecycle. +NOTE: **Note:** +Support for installing the Vault managed application is provided by the GitLab Release Management group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Release Management group](https://about.gitlab.com/handbook/product/product-categories/#release-management-group). + ### Install JupyterHub using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/40) in GitLab 12.8. @@ -1124,6 +1276,10 @@ Refer to the [chart reference](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html) for the available configuration options. +NOTE: **Note:** +Support for installing the JupyterHub managed application is provided by the GitLab Configure group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). + ### Install Elastic Stack using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25138) in GitLab 12.8. @@ -1151,6 +1307,10 @@ available configuration options. NOTE: **Note:** In this alpha implementation of installing Elastic Stack through CI, reading the environment logs through Elasticsearch is unsupported. This is supported if [installed via the UI](#elastic-stack). +NOTE: **Note:** +Support for installing the Elastic Stack managed application is provided by the GitLab APM group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). + ### Install Crossplane using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35675) in GitLab 12.9. @@ -1177,6 +1337,10 @@ management project. Refer to the [chart](https://github.com/crossplane/crossplane/tree/master/cluster/charts/crossplane#configuration) for the available configuration options. Note that this link points to the documentation for the current development release, which may differ from the version you have installed. +NOTE: **Note:** +Support for the Crossplane managed application is provided by the Crossplane team. +If you run into issues, please [open a support ticket](https://github.com/crossplane/crossplane/issues/new/choose) directly. + ### Install Fluentd using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/76) in GitLab 12.10. @@ -1201,6 +1365,10 @@ The configuration chart link points to the current development release, which may differ from the version you have installed. To ensure compatibility, switch to the specific branch or tag you are using. +NOTE: **Note:** +Support for installing the Fluentd managed application is provided by the GitLab Container Security group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). + ### Install Knative using GitLab CI/CD To install Knative, define the `.gitlab/managed-apps/config.yaml` file @@ -1223,6 +1391,10 @@ domain: 'my.wildcard.A.record.dns' If you plan to use GitLab Serverless capabilities, be sure to set an A record wildcard domain on your custom configuration. +NOTE: **Note:** +Support for installing the Knative managed application is provided by the GitLab Configure group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). + #### Knative Metrics GitLab provides [Invocation Metrics](../project/clusters/serverless/index.md#invocation-metrics) for your functions. To collect these metrics, you must have: @@ -1243,6 +1415,92 @@ by running the following command: kubectl delete -f https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/raw/02c8231e30ef5b6725e6ba368bc63863ceb3c07d/src/default-data/knative/istio-metrics.yaml ``` +### Install AppArmor using GitLab CI/CD + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/100) in GitLab 13.1. + +To install AppArmor into the `gitlab-managed-apps` namespace of your cluster using GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: + +```yaml +apparmor: + installed: true +``` + +You can define one or more AppArmor profiles by adding them into `.gitlab/managed-apps/apparmor/values.yaml` as the following: + +```yaml +profiles: + profile-one: |- + profile profile-one { + file, + } +``` + +Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for more information on this chart. + +#### Using AppArmor profiles in your deployments + +After installing AppAmor, you can use profiles by adding Pod Annotations. If you're using Auto +DevOps, you can [customize `auto-deploy-values.yaml`](../../topics/autodevops/customize.md#customize-values-for-helm-chart) +to annotate your pods. Although it's helpful to be aware of the [list of custom attributes](https://gitlab.com/gitlab-org/charts/auto-deploy-app#gitlabs-auto-deploy-helm-chart), you're only required to set +`podAnnotations` as follows: + +```yaml +podAnnotations: + container.apparmor.security.beta.kubernetes.io/auto-deploy-app: localhost/profile-one +``` + +The only information to be changed here is the profile name which is `profile-one` in this example. Refer to the [AppArmor tutorial](https://kubernetes.io/docs/tutorials/clusters/apparmor/#securing-a-pod) for more information on how AppArmor is integrated in Kubernetes. + +#### Using PodSecurityPolicy in your deployments + +NOTE: **Note:** +To enable AppArmor annotations on a Pod Security Policy you must first +load the correspondingAppArmor profile. + +[Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/)are +resources at the cluster level that control security-related +properties of deployed pods. You can use such a policy to enable +loaded AppArmor profiles and apply necessary pod restrictions across a +cluster. You can deploy a new policy by adding the following +to`.gitlab/managed-apps/apparmor/values.yaml`: + +```yaml +securityPolicies: + example: + defaultProfile: profile-one + allowedProfiles: + - profile-one + - profile-two + spec: + privileged: false + seLinux: + rule: RunAsAny + supplementalGroups: + rule: RunAsAny + runAsUser: + rule: RunAsAny + fsGroup: + rule: RunAsAny + volumes: + - '*' +``` + +This example creates a single policy named `example` with the provided +specification, and enables [AppArmor +annotations](https://kubernetes.io/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations)on +it. + +NOTE: **Note:** +Support for installing the AppArmor managed application is provided by the GitLab Container Security group. +If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). + +## Browse applications logs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36769) in GitLab 13.2. + +Logs produced by pods running **GitLab Managed Apps** can be browsed using [**Log Explorer**](../project/clusters/kubernetes_pod_logs.md). + ## Upgrading applications > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24789) in GitLab 11.8. @@ -1342,3 +1600,16 @@ The number and size of nodes might not have enough IP addresses to run or instal For reference, all the AWS instance IP limits are found [in this AWS repository on GitHub](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/pkg/awsutils/vpc_ip_resource_limit.go) (search for `InstanceENIsAvailable`). + +### Unable to install Prometheus + +Installing Prometheus is failing with the following error: + +```shell +# kubectl -n gitlab-managed-apps logs install-prometheus +... +Error: Could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: admission.certmanager.k8s.io/v1beta1: the server is currently unable to handle the request +``` + +This is a bug that was introduced in Helm `2.15` and fixed in `3.0.2`. As a workaround, you'll need +to make sure that [`cert-manager`](#cert-manager) is installed successfully prior to installing Prometheus. diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index 3a430ad55bd..b30ebc57338 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -6,17 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Crossplane configuration -Once Crossplane [is installed](applications.md#crossplane), it must be configured for -use. - +After [installing](applications.md#crossplane) Crossplane, you must configure it for use. The process of configuring Crossplane includes: -1. Configuring RBAC permissions. -1. Configuring Crossplane with a cloud provider. -1. Configure managed service access. -1. Setting up Resource classes. -1. Using Auto DevOps configuration options. -1. Connect to the PostgreSQL instance. +1. [Configure RBAC permissions](#configure-rbac-permissions). +1. [Configure Crossplane with a cloud provider](#configure-crossplane-with-a-cloud-provider). +1. [Configure managed service access](#configure-managed-service-access). +1. [Set up Resource classes](#setting-up-resource-classes). +1. Use [Auto DevOps configuration options](#auto-devops-configuration-options). +1. [Connect to the PostgreSQL instance](#connect-to-the-postgresql-instance). To allow Crossplane to provision cloud services such as PostgreSQL, the cloud provider stack must be configured with a user account. For example: @@ -24,14 +22,13 @@ stack must be configured with a user account. For example: - A service account for GCP. - An IAM user for AWS. -Important notes: +Some important notes: -- This guide uses GCP as an example. However, the process for AWS and Azure will be -similar. -- Crossplane requires the Kubernetes cluster to be VPC native with Alias IPs enabled so -that the IP address of the pods are routable within the GCP network. +- This guide uses GCP as an example, but the processes for AWS and Azure are similar. +- Crossplane requires the Kubernetes cluster to be VPC native with Alias IPs enabled, + so the IP addresses of the pods can be routed within the GCP network. -First, we need to declare some environment variables with configuration that will be used throughout this guide: +First, declare some environment variables with configuration for use in this guide: ```shell export PROJECT_ID=crossplane-playground # the GCP project where all resources reside. @@ -41,228 +38,223 @@ export REGION=us-central1 # the GCP region where the GKE cluster is provisioned. ## Configure RBAC permissions -- For GitLab-managed clusters, RBAC is configured automatically. - -- For non-GitLab managed clusters, ensure that the service account for the token provided can manage resources in the `database.crossplane.io` API group: - - 1. Save the following YAML as `crossplane-database-role.yaml`: - - ```yaml - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRole - metadata: - name: crossplane-database-role - labels: - rbac.authorization.k8s.io/aggregate-to-edit: "true" - rules: - - apiGroups: - - database.crossplane.io - resources: - - postgresqlinstances - verbs: - - get - - list - - create - - update - - delete - - patch - - watch - ``` - - 1. Apply the cluster role to the cluster: - - ```shell - kubectl apply -f crossplane-database-role.yaml - ``` +For GitLab-managed clusters, role-based access control (RBAC) is configured automatically. + +For non-GitLab managed clusters, ensure that the service account for the token +provided can manage resources in the `database.crossplane.io` API group: + +1. Save the following YAML as `crossplane-database-role.yaml`: + + ```yaml + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: crossplane-database-role + labels: + rbac.authorization.k8s.io/aggregate-to-edit: "true" + rules: + - apiGroups: + - database.crossplane.io + resources: + - postgresqlinstances + verbs: + - get + - list + - create + - update + - delete + - patch + - watch + ``` + +1. Apply the cluster role to the cluster: + + ```shell + kubectl apply -f crossplane-database-role.yaml + ``` ## Configure Crossplane with a cloud provider See [Configure Your Cloud Provider Account](https://crossplane.github.io/docs/v0.4/cloud-providers.html) to configure the installed cloud provider stack with a user account. -Note that the Secret and the Provider resource referencing the Secret needs to be +NOTE: **Note:** +The Secret, and the Provider resource referencing the Secret, must be applied to the `gitlab-managed-apps` namespace in the guide. Make sure you change that while following the process. -[Configure Providers](https://crossplane.github.io/docs/v0.4/cloud-providers.html) - ## Configure Managed Service Access -We need to configure connectivity between the PostgreSQL database and the GKE cluster. -This can done by either: +Next, configure connectivity between the PostgreSQL database and the GKE cluster +by either: - Using Crossplane as demonstrated below. - Directly in the GCP console by -[configuring private services access](https://cloud.google.com/vpc/docs/configure-private-services-access). -Create a GlobalAddress and Connection resources: - -```shell -cat > network.yaml <<EOF ---- -# gitlab-ad-globaladdress defines the IP range that will be allocated for cloud services connecting to the instances in the given Network. - -apiVersion: compute.gcp.crossplane.io/v1alpha3 -kind: GlobalAddress -metadata: - name: gitlab-ad-globaladdress -spec: - providerRef: - name: gcp-provider - reclaimPolicy: Delete - name: gitlab-ad-globaladdress - purpose: VPC_PEERING - addressType: INTERNAL - prefixLength: 16 - network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME ---- -# gitlab-ad-connection is what allows cloud services to use the allocated GlobalAddress for communication. Behind -# the scenes, it creates a VPC peering to the network that those service instances actually live. - -apiVersion: servicenetworking.gcp.crossplane.io/v1alpha3 -kind: Connection -metadata: - name: gitlab-ad-connection -spec: - providerRef: - name: gcp-provider - reclaimPolicy: Delete - parent: services/servicenetworking.googleapis.com - network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - reservedPeeringRangeRefs: - - name: gitlab-ad-globaladdress -EOF -``` - -Apply the settings specified in the file with the following command: - -```shell -kubectl apply -f network.yaml -``` - -You can verify creation of the network resources with the following commands. -Verify that the status of both of these resources is ready and is synced. - -```shell -kubectl describe connection.servicenetworking.gcp.crossplane.io gitlab-ad-connection -kubectl describe globaladdress.compute.gcp.crossplane.io gitlab-ad-globaladdress -``` + [configuring private services access](https://cloud.google.com/vpc/docs/configure-private-services-access). + +1. Run the following command, which creates a `network.yaml` file, and configures + `GlobalAddress` and connection resources: + + ```plaintext + cat > network.yaml <<EOF + --- + # gitlab-ad-globaladdress defines the IP range that will be allocated + # for cloud services connecting to the instances in the given Network. + + apiVersion: compute.gcp.crossplane.io/v1alpha3 + kind: GlobalAddress + metadata: + name: gitlab-ad-globaladdress + spec: + providerRef: + name: gcp-provider + reclaimPolicy: Delete + name: gitlab-ad-globaladdress + purpose: VPC_PEERING + addressType: INTERNAL + prefixLength: 16 + network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME + --- + # gitlab-ad-connection is what allows cloud services to use the allocated + # GlobalAddress for communication. Behind the scenes, it creates a VPC peering + # to the network that those service instances actually live. + + apiVersion: servicenetworking.gcp.crossplane.io/v1alpha3 + kind: Connection + metadata: + name: gitlab-ad-connection + spec: + providerRef: + name: gcp-provider + reclaimPolicy: Delete + parent: services/servicenetworking.googleapis.com + network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME + reservedPeeringRangeRefs: + - name: gitlab-ad-globaladdress + EOF + ``` + +1. Apply the settings specified in the file with the following command: + + ```shell + kubectl apply -f network.yaml + ``` + +1. Verify the creation of the network resources, and that both resources are ready and synced. + + ```shell + kubectl describe connection.servicenetworking.gcp.crossplane.io gitlab-ad-connection + kubectl describe globaladdress.compute.gcp.crossplane.io gitlab-ad-globaladdress + ``` ## Setting up Resource classes -Resource classes are a way of defining a configuration for the required managed service. We will define the PostgreSQL Resource class - -- Define a `gcp-postgres-standard.yaml` resource class which contains - -1. A default CloudSQLInstanceClass. -1. A CloudSQLInstanceClass with labels. - -```shell -cat > gcp-postgres-standard.yaml <<EOF -apiVersion: database.gcp.crossplane.io/v1beta1 -kind: CloudSQLInstanceClass -metadata: - name: cloudsqlinstancepostgresql-standard - labels: - gitlab-ad-demo: "true" -specTemplate: - writeConnectionSecretsToNamespace: gitlab-managed-apps - forProvider: - databaseVersion: POSTGRES_11_7 - region: $REGION - settings: - tier: db-custom-1-3840 - dataDiskType: PD_SSD - dataDiskSizeGb: 10 - ipConfiguration: - privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - # this should match the name of the provider created in the above step - providerRef: - name: gcp-provider - reclaimPolicy: Delete ---- -apiVersion: database.gcp.crossplane.io/v1beta1 -kind: CloudSQLInstanceClass -metadata: - name: cloudsqlinstancepostgresql-standard-default - annotations: - resourceclass.crossplane.io/is-default-class: "true" -specTemplate: - writeConnectionSecretsToNamespace: gitlab-managed-apps - forProvider: - databaseVersion: POSTGRES_11_7 - region: $REGION - settings: - tier: db-custom-1-3840 - dataDiskType: PD_SSD - dataDiskSizeGb: 10 - ipConfiguration: - privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - # this should match the name of the provider created in the above step - providerRef: - name: gcp-provider - reclaimPolicy: Delete -EOF -``` - -Apply the resource class configuration with the following command: - -```shell -kubectl apply -f gcp-postgres-standard.yaml -``` - -Verify creation of the Resource class with the following command: - -```shell -kubectl get cloudsqlinstanceclasses -``` - -The Resource Classes allow you to define classes of service for a managed service. We could create another `CloudSQLInstanceClass` which requests for a larger or a faster disk. It could also request for a specific version of the database. +Use resource classes to define a configuration for the required managed service. +This example defines the PostgreSQL Resource class: + +1. Run the following command, which define a `gcp-postgres-standard.yaml` resource + class containing a default `CloudSQLInstanceClass` with labels: + + ```plaintext + cat > gcp-postgres-standard.yaml <<EOF + apiVersion: database.gcp.crossplane.io/v1beta1 + kind: CloudSQLInstanceClass + metadata: + name: cloudsqlinstancepostgresql-standard + labels: + gitlab-ad-demo: "true" + specTemplate: + writeConnectionSecretsToNamespace: gitlab-managed-apps + forProvider: + databaseVersion: POSTGRES_11_7 + region: $REGION + settings: + tier: db-custom-1-3840 + dataDiskType: PD_SSD + dataDiskSizeGb: 10 + ipConfiguration: + privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME + # this should match the name of the provider created in the above step + providerRef: + name: gcp-provider + reclaimPolicy: Delete + --- + apiVersion: database.gcp.crossplane.io/v1beta1 + kind: CloudSQLInstanceClass + metadata: + name: cloudsqlinstancepostgresql-standard-default + annotations: + resourceclass.crossplane.io/is-default-class: "true" + specTemplate: + writeConnectionSecretsToNamespace: gitlab-managed-apps + forProvider: + databaseVersion: POSTGRES_11_7 + region: $REGION + settings: + tier: db-custom-1-3840 + dataDiskType: PD_SSD + dataDiskSizeGb: 10 + ipConfiguration: + privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME + # this should match the name of the provider created in the above step + providerRef: + name: gcp-provider + reclaimPolicy: Delete + EOF + ``` + +1. Apply the resource class configuration with the following command: + + ```shell + kubectl apply -f gcp-postgres-standard.yaml + ``` + +1. Verify creation of the Resource class with the following command: + + ```shell + kubectl get cloudsqlinstanceclasses + ``` + +The Resource Classes allow you to define classes of service for a managed service. +We could create another `CloudSQLInstanceClass` which requests for a larger or a +faster disk. It could also request for a specific version of the database. ## Auto DevOps Configuration Options -The Auto DevOps pipeline can be run with the following options: - -The Environment variables, `AUTO_DEVOPS_POSTGRES_MANAGED` and `AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR` need to be set to provision PostgreSQL using Crossplane - -Alternatively, the following options can be overridden from the values for the Helm chart. - -- `postgres.managed` set to true which will select a default resource class. - The resource class needs to be marked with the annotation - `resourceclass.crossplane.io/is-default-class: "true"`. The CloudSQLInstanceClass - `cloudsqlinstancepostgresql-standard-default` will be used to satisfy the claim. - -- `postgres.managed` set to `true` with `postgres.managedClassSelector` - providing the resource class to choose based on labels. In this case, the - value of `postgres.managedClassSelector.matchLabels.gitlab-ad-demo="true"` - will select the CloudSQLInstance class `cloudsqlinstancepostgresql-standard` - to satisfy the claim request. +You can run the Auto DevOps pipeline with either of the following options: + +- Setting the Environment variables `AUTO_DEVOPS_POSTGRES_MANAGED` and + `AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR` to provision PostgreSQL using Crossplane. +- Overriding values for the Helm chart: + - Set `postgres.managed` to `true`, which selects a default resource class. + Mark the resource class with the annotation + `resourceclass.crossplane.io/is-default-class: "true"`. The CloudSQLInstanceClass + `cloudsqlinstancepostgresql-standard-default` is used to satisfy the claim. + - Set `postgres.managed` to `true` with `postgres.managedClassSelector` + providing the resource class to choose, based on labels. In this case, the + value of `postgres.managedClassSelector.matchLabels.gitlab-ad-demo="true"` + selects the CloudSQLInstance class `cloudsqlinstancepostgresql-standard` + to satisfy the claim request. The Auto DevOps pipeline should provision a PostgresqlInstance when it runs successfully. -Verify creation of the PostgreSQL Instance. +To verify the PostgreSQL instance was created, run this command. When the `STATUS` +field of the PostgresqlInstance changes to `BOUND`, it's successfully provisioned: ```shell -kubectl get postgresqlinstance -``` +$ kubectl get postgresqlinstance -Sample Output: The `STATUS` field of the PostgresqlInstance transitions to `BOUND` when it is successfully provisioned. - -```plaintext NAME STATUS CLASS-KIND CLASS-NAME RESOURCE-KIND RESOURCE-NAME AGE staging-test8 Bound CloudSQLInstanceClass cloudsqlinstancepostgresql-standard CloudSQLInstance xp-ad-demo-24-staging-staging-test8-jj55c 9m ``` -The endpoint of the PostgreSQL instance, and the user credentials, are present in a secret called `app-postgres` within the same project namespace. - -Verify the secret with the database information is created with the following command: +The endpoint of the PostgreSQL instance, and the user credentials, are present in +a secret called `app-postgres` within the same project namespace. You can verify the +secret with the following command: ```shell -kubectl describe secret app-postgres -``` - -Sample Output: +$ kubectl describe secret app-postgres -```plaintext Name: app-postgres Namespace: xp-ad-demo-24-staging Labels: <none> diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index c8755af29a3..892d2bce184 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -97,7 +97,7 @@ Development, Staging, and Production cluster respectively. ```yaml stages: -- deploy + - deploy configure development cluster: stage: deploy diff --git a/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_10.png b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_10.png Binary files differdeleted file mode 100644 index 466552f746e..00000000000 --- a/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_10.png +++ /dev/null diff --git a/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_2.png b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_2.png Binary files differnew file mode 100644 index 00000000000..e1edfcdd024 --- /dev/null +++ b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_2.png diff --git a/doc/user/compliance/compliance_dashboard/index.md b/doc/user/compliance/compliance_dashboard/index.md index 08a26a45b17..e7db73e25d9 100644 --- a/doc/user/compliance/compliance_dashboard/index.md +++ b/doc/user/compliance/compliance_dashboard/index.md @@ -17,7 +17,7 @@ for merging into production. To access the Compliance Dashboard for a group, navigate to **{shield}** **Security & Compliance > Compliance** on the group's menu. - + ## Use cases @@ -27,6 +27,7 @@ You can use the dashboard to: - Get an overview of the latest Merge Request for each project. - See if Merge Requests were approved and by whom. +- See Merge Request authors. - See the latest [CI Pipeline](../../../ci/pipelines/index.md) result for each Merge Request. ## Permissions diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png Binary files differdeleted file mode 100644 index 8070e2cb1a5..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_2.png b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_2.png Binary files differnew file mode 100644 index 00000000000..2d5946503cf --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_2.png diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png Binary files differdeleted file mode 100644 index 741d1237751..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png Binary files differnew file mode 100644 index 00000000000..ee3bb310c1a --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 4ceb393af8c..fb287fb2bf6 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -23,8 +23,8 @@ GitLab checks the License Compliance report, compares the licenses between the source and target branches, and shows the information right on the merge request. Denied licenses will be clearly visible with an `x` red icon next to them as well as new licenses which need a decision from you. In addition, you can -[manually allow or deny](#project-policies-for-license-compliance) -licenses in your project's settings. +[manually allow or deny](#policies) +licenses in your project's license compliance policy section. NOTE: **Note:** If the license compliance report doesn't have anything to compare to, no information @@ -46,7 +46,7 @@ When GitLab detects a **Denied** license, you can view it in the [license list]( You can view and modify existing policies from the [policies](#policies) tab. - + ## Use cases @@ -64,7 +64,7 @@ The following languages and package managers are supported. | Go | [Godep](https://github.com/tools/godep), [go mod](https://github.com/golang/go/wiki/Modules) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Java | [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| | .NET | [Nuget](https://www.nuget.org/) (.NET Framework is supported via the [mono project](https://www.mono-project.com/). Windows specific dependencies are not supported at this time.) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Python | [pip](https://pip.pypa.io/en/stable/) (Python is supported through [requirements.txt](https://pip.pypa.io/en/1.1/requirements/) and [Pipfile.lock](https://github.com/pypa/pipfile#pipfilelock).) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Python | [pip](https://pip.pypa.io/en/stable/) (Python is supported through [requirements.txt](https://pip.pypa.io/en/stable/user_guide/#requirements-files) and [Pipfile.lock](https://github.com/pypa/pipfile#pipfilelock).) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Ruby | [gem](https://rubygems.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage) |[License Finder](https://github.com/pivotal/LicenseFinder)| @@ -86,7 +86,7 @@ which means that the reported licenses might be incomplete or inaccurate. | Elixir | [mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) |[License Finder](https://github.com/pivotal/LicenseFinder)| | C++/C | [conan](https://conan.io/) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Scala | [sbt](https://www.scala-sbt.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Rust | [cargo](https://crates.io/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Rust | [cargo](https://crates.io) |[License Finder](https://github.com/pivotal/LicenseFinder)| | PHP | [composer](https://getcomposer.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| ## Requirements @@ -339,7 +339,7 @@ strict-ssl = false ### Configuring Yarn projects -You can configure Yarn projects by using a [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc/) +You can configure Yarn projects by using a [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc) file. #### Using private Yarn registries @@ -385,6 +385,26 @@ You can supply a custom root certificate to complete TLS verification by using t specifying a `ca` setting in a [`.bowerrc`](https://bower.io/docs/config/#bowerrc-specification) file. +#### Using private Bundler registries + +If you have a private Bundler registry you can use the +[`source`](https://bundler.io/man/gemfile.5.html#GLOBAL-SOURCES) +setting to specify its location. + +For example: + +```plaintext +source "https://gems.example.com" +``` + +#### Custom root certificates for Bundler + +You can supply a custom root certificate to complete TLS verification by using the +`ADDITIONAL_CA_CERT_BUNDLE` [environment variable](#available-variables), or by +specifying a [`BUNDLE_SSL_CA_CERT`](https://bundler.io/v2.0/man/bundle-config.1.html) +[environment variable](../../../ci/variables/README.md#custom-environment-variables) +in the job definition. + ### Configuring Conan projects You can configure [Conan](https://conan.io/) projects by adding a `.conan` directory to your @@ -490,6 +510,29 @@ license_scanning: GOFLAGS: '-insecure' ``` +#### Using private NuGet registries + +If you have a private NuGet registry you can add it as a source +by adding it to the [`packageSources`](https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file#package-source-sections) +section of a [`nuget.config`](https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file) file. + +For example: + +```xml +<?xml version="1.0" encoding="utf-8"?> +<configuration> + <packageSources> + <clear /> + <add key="custom" value="https://nuget.example.com/v3/index.json" /> + </packageSources> +</configuration> +``` + +#### Custom root certificates for NuGet + +You can supply a custom root certificate to complete TLS verification by using the +`ADDITIONAL_CA_CERT_BUNDLE` [environment variable](#available-variables). + ### Migration from `license_management` to `license_scanning` In GitLab 12.8 a new name for `license_management` job was introduced. This change was made to improve clarity around the purpose of the scan, which is to scan and collect the types of licenses present in a projects dependencies. @@ -594,6 +637,7 @@ your code and generate security reports, without requiring internet access. Additional configuration may be needed for connecting to [private Bower registries](#using-private-bower-registries), +[private Bundler registries](#using-private-bundler-registries), [private Conan registries](#using-private-bower-registries), [private Go registries](#using-private-go-registries), [private Maven repositories](#using-private-maven-repos), @@ -601,69 +645,9 @@ Additional configuration may be needed for connecting to [private Python repositories](#using-private-python-repos), and [private Yarn registries](#using-private-yarn-registries). -Exact name matches are required for [project policies](#project-policies-for-license-compliance) +Exact name matches are required for [project policies](#policies) when running in an offline environment ([see related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/212388)). -## Project policies for License Compliance - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5940) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. - -From the project's settings: - -- The list of licenses and their status can be managed. -- Licenses can be manually allowed or denied. - -To allow or deny a license: - -1. Either use the **Manage licenses** button in the merge request widget, or - navigate to the project's **Settings > CI/CD** and expand the - **License Compliance** section. -1. Click the **Add a license** button. - -  - -1. In the **License name** dropdown, either: - - Select one of the available licenses. You can search for licenses in the field - at the top of the list. - - Enter arbitrary text in the field at the top of the list. This will cause the text to be - added as a license name to the list. -1. Select the **Allow** or **Deny** radio button to allow or deny respectively - the selected license. - -To modify an existing license: - -1. In the **License Compliance** list, click the **Allow/Deny** dropdown to change it to the desired status. - -  - -Searching for Licenses: - -1. Use the **Search** box to search for a specific license. - -  - -## License Compliance report under pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5491) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2. - -From your project's left sidebar, navigate to **CI/CD > Pipelines** and click on the -pipeline ID that has a `license_scanning` job to see the Licenses tab with the listed -licenses (if any). - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> - ## License list > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13582) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.7. @@ -696,13 +680,40 @@ and the associated classifications for each. Policies can be configured by maintainers of the project. - - + + Developers of the project can view the policies configured in a project.  +### Enabling License Approvals within a project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. + +`License-Check` is an approval rule you can enable to allow an approver, individual, or group to +approve a merge request that contains a `denied` license. + +You can enable `License-Check` one of two ways: + +- Create a [project approval rule](../../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) + with the case-sensitive name `License-Check`. +- Create an approval group in the [project policies section for License Compliance](#policies). + You must set this approval group's number of approvals required to greater than zero. Once you + enable this group in your project, the approval rule is enabled for all merge requests. + +Any code changes cause the approvals required to reset. + +An approval is required when a license report: + +- Contains a dependency that includes a software license that is `denied`. +- Is not generated during pipeline execution. + +An approval is optional when a license report: + +- Contains no software license violations. +- Contains only new licenses that are `allowed` or unknown. + ## Troubleshooting ### `ERROR -- : asdf: No preset version installed for command` diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 44802214d7b..599f46b2c40 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -487,14 +487,15 @@ For example, to customize the commit message to output NOTE: **Note:** Custom commit messages for each applied Suggestion (and for batch Suggestions) will be -introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/issues/25381). +introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/-/issues/25381). ### Batch Suggestions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/25486) in GitLab 13.1 as an [alpha feature](https://about.gitlab.com/handbook/product/#alpha). -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-batch-suggestions). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [alpha feature](https://about.gitlab.com/handbook/product/#alpha). +> - It was deployed behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) on GitLab 13.2. +> - It's enabled on GitLab.com. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-batch-suggestions-core-only). You can apply multiple suggestions at once to reduce the number of commits added to your branch to address your reviewers' requests. @@ -515,25 +516,25 @@ to your branch to address your reviewers' requests.  -#### Enable or disable Batch Suggestions +#### Enable or disable Batch Suggestions **(CORE ONLY)** Batch Suggestions is -deployed behind a feature flag that is **disabled by default**. +deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it for your instance. +can opt to disable it for your instance. To enable it: ```ruby # Instance-wide -Feature.enable(:batched_suggestions) +Feature.enable(:batch_suggestions) ``` To disable it: ```ruby # Instance-wide -Feature.disable(:batched_suggestions) +Feature.disable(:batch_suggestions) ``` ## Start a thread by replying to a standard comment diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 6522f5c4053..bcaba97cab7 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -78,31 +78,34 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | Setting | GitLab.com | Default | | ----------- | ----------------- | ------------- | | Artifacts maximum size (uncompressed) | 1G | 100M | -| Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | kept forever | deleted after 30 days unless otherwise specified | +| Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | | Scheduled Pipeline Cron | `*/5 * * * *` | `19 * * * *` | | [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited | [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited | | [Max number of instance level variables](../../administration/instance_limits.md#number-of-instance-level-variables) | `25` | `25` | +| [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs-core-only) | 3 months | Never | ## Repository size limit -The maximum size your Git repository is allowed to be, including LFS. If you are near -or over the size limit, you can [reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). +GitLab.com has the following [account limits](../admin_area/settings/account_and_limit_settings.md) enabled. If a setting is not listed, it is set to the default value. -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ------------- | -| Repository size including LFS | 10G | Unlimited | +If you are near +or over the repository size limit, you can [reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). + +| Setting | GitLab.com | Default | +| ----------- | ----------- | ------------- | +| Repository size including LFS | 10 GB | Unlimited | NOTE: **Note:** -`git push` and GitLab project imports are limited to 5GB per request. Git LFS and imports other than a file upload are not affected by this limit. +`git push` and GitLab project imports are limited to 5 GB per request through Cloudflare. Git LFS and imports other than a file upload are not affected by this limit. ## IP range GitLab.com is using the IP range `34.74.90.64/28` for traffic from its Web/API fleet. This whole range is solely allocated to GitLab. You can expect connections from webhooks or repository mirroring to come -from those IPs and whitelist them. +from those IPs and allow them. -GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com you might need to whitelist CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)) +GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)). For outgoing connections from CI/CD runners we are not providing static IP addresses. All our runners are deployed into Google Cloud Platform (GCP) - any IP based @@ -334,9 +337,9 @@ Windows Shared Runners: ```yaml .shared_windows_runners: tags: - - shared-windows - - windows - - windows-1809 + - shared-windows + - windows + - windows-1809 stages: - build @@ -349,17 +352,17 @@ before_script: build: extends: - - .shared_windows_runners + - .shared_windows_runners stage: build script: - - echo "running scripts in the build job" + - echo "running scripts in the build job" test: extends: - - .shared_windows_runners + - .shared_windows_runners stage: test script: - - echo "running scripts in the test job" + - echo "running scripts in the test job" ``` #### Limitations and known issues @@ -581,9 +584,9 @@ is used to forward logs to an [Elastic cluster](https://gitlab.com/gitlab-com/ru You can view more information in our runbooks such as: -- A [detailed list of what we're logging](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#what-are-we-logging) -- Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#retention) -- A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#logging-infrastructure-overview) +- A [detailed list of what we're logging](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#what-are-we-logging) +- Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#retention) +- A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#logging-infrastructure-overview) ## GitLab.com at scale diff --git a/doc/user/group/bulk_editing/img/bulk-editing.png b/doc/user/group/bulk_editing/img/bulk-editing.png Binary files differdeleted file mode 100644 index ca1662a781b..00000000000 --- a/doc/user/group/bulk_editing/img/bulk-editing.png +++ /dev/null diff --git a/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png b/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png Binary files differnew file mode 100644 index 00000000000..9f28fabdf0c --- /dev/null +++ b/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png diff --git a/doc/user/group/bulk_editing/index.md b/doc/user/group/bulk_editing/index.md index c4ccc1f7b52..35bdc6696eb 100644 --- a/doc/user/group/bulk_editing/index.md +++ b/doc/user/group/bulk_editing/index.md @@ -8,12 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: **Note:** Bulk editing issues and merge requests is also available at the **project level**. -For more details, see [Bulk editing issues, epics, and merge requests](../../project/bulk_editing.md). +For more details, see [Bulk editing issues and merge requests at the project level](../../project/bulk_editing.md). If you want to update attributes across multiple issues, epics, or merge requests in a group, you can do it by bulk editing them, that is, editing them together. - + ## Bulk edit issues at the group level @@ -24,8 +24,12 @@ You need a permission level of [Reporter or higher](../../permissions.md) to man When bulk editing issues in a group, you can edit the following attributes: +- Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in + [GitLab Premium](https://about.gitlab.com/pricing/) 13.2.) **(PREMIUM)** - Milestone - Labels +- Health status ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218395) in + [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2.) **(ULTIMATE)** To update multiple project issues at the same time: diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 5cdac7ae892..89e0c4898fb 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -38,10 +38,11 @@ the project. In the case of sub-groups, GitLab uses the cluster of the closest ancestor group to the project, provided the cluster is not disabled. -## Multiple Kubernetes clusters **(PREMIUM)** +## Multiple Kubernetes clusters -With [GitLab Premium](https://about.gitlab.com/pricing/premium/), you can associate -more than one Kubernetes cluster to your group, and maintain different clusters +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab Core in 13.2. + +You can associate more than one Kubernetes cluster to your group, and maintain different clusters for different environments, such as development, staging, and production. When adding another cluster, @@ -93,7 +94,7 @@ To clear the cache: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8. Domains at the cluster level permit support for multiple domains -per [multiple Kubernetes clusters](#multiple-kubernetes-clusters-premium). When specifying a domain, +per [multiple Kubernetes clusters](#multiple-kubernetes-clusters) When specifying a domain, this will be automatically set as an environment variable (`KUBE_INGRESS_BASE_DOMAIN`) during the [Auto DevOps](../../../topics/autodevops/index.md) stages. @@ -127,8 +128,8 @@ And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/RE ```yaml stages: -- test -- deploy + - test + - deploy test: stage: test diff --git a/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png b/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png Binary files differnew file mode 100644 index 00000000000..c7e1448fea9 --- /dev/null +++ b/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png diff --git a/doc/user/group/epics/img/epics_list_view_v12.5.png b/doc/user/group/epics/img/epics_list_view_v12.5.png Binary files differdeleted file mode 100644 index 6e3c39009be..00000000000 --- a/doc/user/group/epics/img/epics_list_view_v12.5.png +++ /dev/null diff --git a/doc/user/group/epics/img/new_epic_form_v13.2.png b/doc/user/group/epics/img/new_epic_form_v13.2.png Binary files differnew file mode 100644 index 00000000000..3d24763d105 --- /dev/null +++ b/doc/user/group/epics/img/new_epic_form_v13.2.png diff --git a/doc/user/group/epics/img/new_epic_from_groups_v13.2.png b/doc/user/group/epics/img/new_epic_from_groups_v13.2.png Binary files differnew file mode 100644 index 00000000000..85bc4255595 --- /dev/null +++ b/doc/user/group/epics/img/new_epic_from_groups_v13.2.png diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 85164292265..a2b04e2d7fe 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -14,8 +14,15 @@ Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. -<!-- Possibly swap this file with one of a single epic --> - +An epic's page contains the following tabs: + +- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are + shown in a tree view. + - Click the chevron (**>**) next to a parent epic to reveal the child epics and issues. + - Hover over the total counts to see a breakdown of open and closed items. +- **Roadmap**: a roadmap view of child epics which have start and due dates. + + ## Use cases @@ -28,6 +35,7 @@ milestones. To learn what you can do with an epic, see [Manage epics](manage_epics.md). Possible actions include: - [Create an epic](manage_epics.md#create-an-epic) +- [Edit an epic](manage_epics.md#edit-an-epic) - [Bulk-edit epics](../bulk_editing/index.md#bulk-edit-epics) - [Delete an epic](manage_epics.md#delete-an-epic) - [Close an epic](manage_epics.md#close-an-epic) @@ -165,6 +173,19 @@ Once you write your comment, you can either: - Click **Comment**, and your comment will be published. - Click **Start thread**, and you will start a thread within that epic's discussion. +### Activity sort order + +> [Introduced](https://https://gitlab.com/gitlab-org/gitlab/-/issues/214364) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +You can reverse the default order and interact with the activity feed sorted by most recent items +at the top. Your preference is saved via local storage and automatically applied to every issue +you view. + +To change the activity sort order, click the **Oldest first** dropdown menu and select either oldest +or newest items to be shown first. + + + ## Award emoji You can [award an emoji](../../award_emojis.md) to that epic or its comments. diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 26d5cb51e01..4f9bb0e24fb 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -18,12 +18,42 @@ A paginated list of epics is available in each group from where you can create a new epic. The list of epics includes also epics from all subgroups of the selected group. From your group page: -1. Go to **Epics**. +### Create an epic from the epic list + +To create an epic from the epic list, in a group: + +1. Go to **{epic}** **Epics**. 1. Click **New epic**. 1. Enter a descriptive title. 1. Click **Create epic**. -You will be taken to the new epic where can edit the following details: +### Access the New Epic form + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211533) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +There are two ways to get to the New Epic form and create an epic in the group you're in: + +- From an epic in your group, click **New Epic**. +- From anywhere, in the top menu, click **plus** (**{plus-square}**) **> New epic**. + +  + +### Elements of the New Epic form + +When you're creating a new epic, these are the fields you can fill in: + +- Title +- Description +- Confidentiality checkbox +- Labels +- Start date +- Due date + + + +## Edit an epic + +After you create an epic, you can edit change the following details: - Title - Description @@ -31,15 +61,16 @@ You will be taken to the new epic where can edit the following details: - Due date - Labels -An epic's page contains the following tabs: +To edit an epic's title or description: -- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are - shown in a tree view. - - Click the <kbd>></kbd> beside a parent epic to reveal the child epics and issues. - - Hover over the total counts to see a breakdown of open and closed items. -- **Roadmap**: a roadmap view of child epics which have start and due dates. +1. Click the **Edit title and description** **{pencil}** button. +1. Make your changes. +1. Click **Save changes**. - +To edit an epics' start date, due date, or labels: + +1. Click **Edit** next to each section in the epic sidebar. +1. Select the dates or labels for your epic. ## Bulk-edit epics @@ -118,27 +149,17 @@ The sort option and order is saved and used wherever you browse epics, including ## Make an epic confidential -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213068) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-confidential-epics-premium-only). **(PREMIUM ONLY)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213068) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0 behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/224513) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. When you're creating an epic, you can make it confidential by selecting the **Make this epic confidential** checkbox. -### Enable Confidential Epics **(PREMIUM ONLY)** +### Disable confidential epics **(PREMIUM ONLY)** -The Confidential Epics feature is under development and not ready for production use. -It's deployed behind a feature flag that is **disabled by default**. +The confidential epics feature is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. - -To enable it: - -```ruby -Feature.enable(:confidential_epics) -``` +can disable it for your self-managed instance. To disable it: @@ -233,7 +254,7 @@ To move an issue to another epic: > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) to [GitLab Premium](https://about.gitlab.com/pricing/) in 12.8. If you have the necessary [permissions](../../permissions.md) to close an issue and create an -epic in the parent group, you can promote an issue to an epic with the `/promote` +epic in the immediate parent group, you can promote an issue to an epic with the `/promote` [quick action](../../project/quick_actions.md#quick-actions-for-issues-merge-requests-and-epics). Only issues from projects that are in groups can be promoted. When attempting to promote a confidential issue, a warning will display. Promoting a confidential issue to an epic will make all information diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 324c912b2be..5ba0680127c 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -31,7 +31,7 @@ Each group on the **Groups** page is listed with: - How many subgroups it has. - How many projects it contains. -- How many members the group has, not including members inherited from parent groups. +- How many members the group has, not including members inherited from parent group(s). - The group's visibility. - A link to the group's settings, if you have sufficient permissions. - A link to leave the group, if you are a member. @@ -184,6 +184,33 @@ of a group: 1. Give a different member **Owner** permissions. 1. Have the new owner sign in and remove **Owner** permissions from you. +## Remove a member from the group + +Only users with permissions of [Owner](../permissions.md#group-members-permissions) can manage +group members. + +You can remove a member from the group if the given member has a direct membership in the group. If +membership is inherited from a parent group, then the member can be removed only from the parent +group itself. + +When removing a member, you can decide whether to unassign the user from all issues and merge +requests they are currently assigned or leave the assignments as they are. + +- **Unassigning the removed member** from all issues and merge requests might be helpful when a user + is leaving a private group and you wish to revoke their access to any issues and merge requests + they are assigned. +- **Keeping the issues and merge requests assigned** might be helpful for groups that accept public + contributions where a user doesn't have to be a member to be able to contribute to issues and + merge requests. + +To remove a member from a group: + +1. In a group, go to **{users}** **Members**. +1. Click the **Delete** **{remove}** button next to a group member you want to remove. + A **Remove member** modal appears. +1. (Optional) Select the **Also unassign this user from related issues and merge requests** checkbox. +1. Click **Remove member**. + ## Changing the default branch protection of a group > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7583) in GitLab 12.9. @@ -397,7 +424,7 @@ When transferring groups, note: - Changing a group's parent can have unintended side effects. See [Redirects when changing repository paths](../project/index.md#redirects-when-changing-repository-paths). - You can only transfer groups to groups you manage. - You must update your local repositories to point to the new location. -- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. +- If the immediate parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. - Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. ## Group settings @@ -435,7 +462,7 @@ It is currently not possible to rename a namespace if it contains a project with [Container Registry](../packages/container_registry/index.md) tags, because the project cannot be moved. -TIP: **TIP:** +TIP: **Tip:** If you want to retain ownership over the original namespace and protect the URL redirects, then instead of changing a group's path or renaming a username, you can create a new group and transfer projects to it. @@ -516,7 +543,7 @@ underlying projects, issues, etc, by IP address. This can help ensure that particular content doesn't leave the premises, while not blocking off access to the entire instance. -Add one or more allowed IP subnets using CIDR notation in comma separated format to the group settings and anyone +Add one or more allowed IP subnets using CIDR notation to the group settings and anyone coming from a different IP address won't be able to access the restricted content. @@ -529,6 +556,12 @@ Restriction currently applies to: To avoid accidental lock-out, admins and group owners are able to access the group regardless of the IP restriction. +To enable this feature: + +1. Navigate to the group’s **Settings > General** page. +1. Expand the **Permissions, LFS, 2FA** section, and enter IP address ranges into **Allow access to the following IP addresses** field. +1. Click **Save changes**. + #### Allowed domain restriction **(PREMIUM)** >- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7297) in [GitLab Premium and Silver](https://about.gitlab.com/pricing/) 12.2. @@ -554,7 +587,7 @@ Some domains cannot be restricted. These are the most popular public email domai To enable this feature: 1. Navigate to the group's **Settings > General** page. -1. Expand the **Permissions, LFS, 2FA** section, and enter the domain names into **Restrict membership by email** field. You can enter multiple domains by separating each domain with a comma (,). +1. Expand the **Permissions, LFS, 2FA** section, and enter the domain names into **Restrict membership by email** field. 1. Click **Save changes**. This will enable the domain-checking for all new users added to the group from this moment on. @@ -571,9 +604,9 @@ You can only choose projects in the group as the template source. This includes projects shared with the group, but it **excludes** projects in subgroups or parent groups of the group being configured. -You can configure this feature for both subgroups and parent groups. A project +You can configure this feature for both subgroups and immediate parent groups. A project in a subgroup will have access to the templates for that subgroup, as well as -any parent groups. +any immediate parent groups.  @@ -617,6 +650,23 @@ To enable this feature: 1. Expand the **Permissions, LFS, 2FA** section, and select **Disable group mentions**. 1. Click **Save changes**. +#### Enabling delayed Project removal **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. + +By default, projects within a group are deleted immediately. +Optionally, on [Premium or Silver](https://about.gitlab.com/pricing/) or higher tiers, +you can configure the projects within a group to be deleted after a delayed interval. + +During this interval period, the projects will be in a read-only state and can be restored, if required. +The interval period defaults to 7 days, and can be modified by an admin in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). + +To enable delayed deletion of projects: + +1. Navigate to the group's **Settings > General** page. +1. Expand the **Permissions, LFS, 2FA** section, and check **Enable delayed project removal**. +1. Click **Save changes**. + ### Advanced settings - **Projects**: View all projects within that group, add members to each project, diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 2704147dcdd..bc9d228011a 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -8,11 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Iterations **(STARTER)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214713) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.1. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's able to be enabled or disabled per-group -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-iterations-core-only). **(CORE ONLY)** +> - It was deployed behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/221047) on GitLab 13.2. +> - It's enabled on GitLab.com. +> - It's able to be enabled or disabled per-group. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-iterations-core-only). **(CORE ONLY)** Iterations are a way to track issues over a period of time. This allows teams to track velocity and volatility metrics. Iterations can be used with [milestones](../../project/milestones/index.md) @@ -38,7 +39,7 @@ From there you can create a new iteration or click an iteration to get a more de ## Create an iteration NOTE: **Note:** -A permission level of [Developer or higher](../../permissions.md) is required to create iterations. +You need Developer [permissions](../../permissions.md) or higher to create an iteration. To create an iteration: @@ -47,12 +48,20 @@ To create an iteration: 1. Enter the title, a description (optional), a start date, and a due date. 1. Click **Create iteration**. The iteration details page opens. -### Enable Iterations **(CORE ONLY)** +## Edit an iteration -GitLab Iterations feature is under development and not ready for production use. -It is deployed behind a feature flag that is **disabled by default**. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218277) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.2. + +NOTE: **Note:** +You need Developer [permissions](../../permissions.md) or higher to edit an iteration. + +To edit an iteration, click the three-dot menu (**{ellipsis_v}**) > **Edit iteration**. + +## Disable Iterations **(CORE ONLY)** + +GitLab Iterations feature is deployed with a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. `:group_iterations` can be enabled or disabled per-group. +can disable it for your instance. `:group_iterations` can be enabled or disabled per-group. To enable it: diff --git a/doc/user/group/roadmap/img/roadmap_view_v13_0.png b/doc/user/group/roadmap/img/roadmap_view_v13_0.png Binary files differdeleted file mode 100644 index a5b76b84418..00000000000 --- a/doc/user/group/roadmap/img/roadmap_view_v13_0.png +++ /dev/null diff --git a/doc/user/group/roadmap/img/roadmap_view_v13_2.png b/doc/user/group/roadmap/img/roadmap_view_v13_2.png Binary files differnew file mode 100644 index 00000000000..b4f889afaa4 --- /dev/null +++ b/doc/user/group/roadmap/img/roadmap_view_v13_2.png diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index 614ed700cfc..c185055f6b2 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -12,22 +12,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/5164) and later, the epic bars show epics' title, progress, and completed weight percentage. > - Milestones appear in roadmaps in [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/6802), and later. > - Feature flag for milestones visible in roadmaps removed in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29641). +> - In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/214375) and later, the Roadmap also shows milestones in projects in a group. +> - In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/212494) and later, milestone bars can be collapsed and expanded. -Epics and milestones within a group containing **Start date** and/or **Due date** -can be visualized in a form of a timeline (that is, a Gantt chart). The Roadmap page -shows such a visualization for all the epics and milestones which are under a group or one of its -subgroups. +Epics and milestones within a group containing a start date or due date can be visualized in a form +of a timeline (that is, a Gantt chart). The Roadmap page shows the epics and milestones in a +group, one of its subgroups, or a project in one of the groups. On the epic bars, you can see the each epic's title, progress, and completed weight percentage. When you hover over an epic bar, a popover appears with the epic's title, start date, due date, and weight completed. You can expand epics that contain child epics to show their child epics in the roadmap. -You can click the chevron **{chevron-down}** next to the epic title to expand and collapse the child epics. +You can click the chevron (**{chevron-down}**) next to the epic title to expand and collapse the child epics. On top of the milestone bars, you can see their title. When you hover a milestone bar or title, a popover appears with its title, start date and due date. +You can also click the chevron (**{chevron-down}**) next to the **Milestones** heading to toggle the list of the milestone bars. - + A dropdown menu allows you to show only open or closed epics. By default, all epics are shown. diff --git a/doc/user/group/saml_sso/group_managed_accounts.md b/doc/user/group/saml_sso/group_managed_accounts.md new file mode 100644 index 00000000000..e317573d89d --- /dev/null +++ b/doc/user/group/saml_sso/group_managed_accounts.md @@ -0,0 +1,116 @@ +--- +type: reference, howto +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Group Managed Accounts **(PREMIUM)** + +CAUTION: **Warning:** +This is a [Closed Beta](https://about.gitlab.com/handbook/product/#closed-beta) feature. + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/709) in GitLab 12.1. +> - It's deployed behind a feature flag, disabled by default. + +When [SSO for Groups](index.md) is being enforced, groups can enable an additional level of protection by enforcing the creation of dedicated user accounts to access the group. + +With group-managed accounts enabled, users are required to create a new, dedicated user linked to the group. +The notification email address associated with the user is locked to the email address received from the configured identity provider. +Without group-managed accounts, users can link their SAML identity with any existing user on the instance. + +When this option is enabled: + +- All users in the group will be required to log in via the SSO URL associated with the group. +- After the group-managed account has been created, group activity will require the use of this user account. +- Users can't share a project in the group outside the top-level group (also applies to forked projects). + +Upon successful authentication, GitLab prompts the user with options, based on the email address received from the configured identity provider: + +- To create a unique account with the newly received email address. +- If the received email address matches one of the user's verified GitLab email addresses, the option to convert the existing account to a group-managed account. ([Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/13481).) + +Since use of the group-managed account requires the use of SSO, users of group-managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: + +- The user will be unable to access the group (their credentials will no longer work on the identity provider when prompted to SSO). +- Contributions in the group (e.g. issues, merge requests) will remain intact. + +## Assertions + +When using group-managed accounts, the following user details need to be passed to GitLab as SAML +assertions to be able to create a user. + +| Field | Supported keys | +|-----------------|----------------| +| Email (required)| `email`, `mail` | +| Full Name | `name` | +| First Name | `first_name`, `firstname`, `firstName` | +| Last Name | `last_name`, `lastname`, `lastName` | + +## Feature flag **(PREMIUM ONLY)** + +Currently the group-managed accounts feature is behind a feature flag: `group_managed_accounts`. The flag is disabled by default. +To activate the feature, ask a GitLab administrator with Rails console access to run: + +```ruby +Feature.enable(:group_managed_accounts) +``` + +## Project restrictions for Group-managed accounts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12420) in GitLab 12.9. + +Projects within groups with enabled group-managed accounts are not to be shared with: + +- Groups outside of the parent group. +- Members who are not users managed by this group. + +This restriction also applies to projects forked from or to those groups. + +## Outer forks restriction for Group-managed accounts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34648) in GitLab 12.9. + +Groups with group-managed accounts can disallow forking of projects to destinations outside the group. +To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. +When enabled, projects within the group can only be forked to other destinations within the group (including its subgroups). + +## Credentials inventory for Group-managed accounts **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38133) in GitLab 12.8. + +Owners who manage user accounts in a group can view the following details of personal access tokens and SSH keys: + +- Owners +- Scopes +- Usage patterns + +To access the Credentials inventory of a group, navigate to **{shield}** **Security & Compliance > Credentials** in your group's sidebar. + +This feature is similar to the [Credentials inventory for self-managed instances](../../admin_area/credentials_inventory.md). + +## Limiting lifetime of personal access tokens of users in Group-managed accounts **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118893) in GitLab 12.10. + +Users in a group managed account can optionally specify an expiration date for +[personal access tokens](../../profile/personal_access_tokens.md). +This expiration date is not a requirement, and can be set to any arbitrary date. + +Since personal access tokens are the only token needed for programmatic access to GitLab, organizations with security requirements may want to enforce more protection to require regular rotation of these tokens. + +### Setting a limit + +Only a GitLab administrator or an owner of a Group-managed account can set a limit. Leaving it empty means that the [instance level restrictions](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only) on the lifetime of personal access tokens will apply. + +To set a limit on how long personal access tokens are valid for users in a group managed account: + +1. Navigate to the **{settings}** **Settings > General** page in your group's sidebar. +1. Expand the **Permissions, LFS, 2FA** section. +1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. +1. Click **Save changes**. + +Once a lifetime for personal access tokens is set, GitLab will: + +- Apply the lifetime for new personal access tokens, and require users managed by the group to set an expiration date that is no later than the allowed lifetime. +- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the allowed lifetime. Three hours is given to allow administrators/group owner to change the allowed lifetime, or remove it, before revocation takes place. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 81684038dc2..afd676cf897 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -5,32 +5,25 @@ group: Access info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# SAML SSO for GitLab.com groups **(SILVER ONLY)** +# SAML SSO for GitLab.com groups **(PREMIUM)** -> Introduced in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.0. +> Introduced in GitLab 11.0. -SAML on GitLab.com allows users to be added to a group. Those users can then sign in to GitLab.com. If such users don't already have an account on the GitLab instance, they can create one when signing in for the first time. +This page describes SAML for Groups. For instance-wide SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -If you follow our guidance to automate user provisioning using [SCIM](scim_setup.md) or [group-managed accounts](#group-managed-accounts), you do not need to create such accounts manually. +SAML on GitLab.com allows users to sign in through their SAML identity provider. If the user is not already a member, the sign-in process automatically adds the user to the appropriate group. -User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). +If you follow our guidance to automate user provisioning using [SCIM](scim_setup.md) or [group-managed accounts](group_managed_accounts.md), you do not need to create such accounts manually. -## Important notes - -Note the following: - -- This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab - instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -- SAML SSO for GitLab.com groups requires SCIM to sync users between providers. If a - group is not using SCIM, group Owners will still need to manage user accounts (for example, - removing users when necessary). +User synchronization of SAML SSO groups is supported through [SCIM](scim_setup.md). SCIM supports adding and removing users from the GitLab group. +For example, if you remove a user from the SCIM app, SCIM removes that same user from the GitLab group. ## Configuring your Identity Provider 1. Navigate to the group and click **Settings > SAML SSO**. -1. Configure your SAML server using the **Assertion consumer service URL** and **Identifier**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [your identity provider's documentation](#providers) for more details. +1. Configure your SAML server using the **Assertion consumer service URL**, **Identifier**, and **GitLab single sign-on URL**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [specific identity provider documentation](#providers) for more details. 1. Configure the SAML response to include a NameID that uniquely identifies each user. -1. Configure required assertions using the [table below](#assertions). +1. Configure [required assertions](group_managed_accounts.md#assertions) if using [Group Managed Accounts](group_managed_accounts.md). 1. Once the identity provider is set up, move on to [configuring GitLab](#configuring-gitlab).  @@ -55,123 +48,6 @@ Once users have signed into GitLab using the SSO SAML setup, changing the `NameI We recommend setting the NameID format to `Persistent` unless using a field (such as email) that requires a different format. -### SSO enforcement - -- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. -- [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. - -With this option enabled, users must use your group's GitLab single sign on URL to be added to the group or be added via SCIM. Users cannot be added manually, and may only access project/group resources via the UI by signing in through the SSO URL. - -However, users will not be prompted to log via SSO on each visit. GitLab will check whether a user has authenticated through the SSO link, and will only prompt the user to login via SSO if the session has expired. - -We intend to add a similar SSO requirement for [Git and API activity](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in the future. - -When SSO enforcement is enabled for a group, users cannot share a project in the group outside the top-level group, even if the project is forked. - -#### Group-managed accounts - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/709) in GitLab 12.1. - -When SSO is being enforced, groups can enable an additional level of protection by enforcing the creation of dedicated user accounts to access the group. - -Without group-managed accounts, users can link their SAML identity with any existing user on the instance. With group-managed accounts enabled, users are required to create a new, dedicated user linked to the group. The notification email address associated with the user is locked to the email address received from the configured identity provider. - -When this option is enabled: - -- All existing and new users in the group will be required to log in via the SSO URL associated with the group. -- After the group-managed account has been created, group activity will require the use of this user account. -- Users can't share a project in the group outside the top-level group (also applies to forked projects). - -Upon successful authentication, GitLab prompts the user with options, based on the email address received from the configured identity provider: - -- To create a unique account with the newly received email address. -- If the received email address matches one of the user's verified GitLab email addresses, the option to convert the existing account to a group-managed account. ([Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/13481).) - -Since use of the group-managed account requires the use of SSO, users of group-managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: - -- The user will be unable to access the group (their credentials will no longer work on the identity provider when prompted to SSO). -- Contributions in the group (e.g. issues, merge requests) will remain intact. - -##### Feature flag - -Currently the group-managed accounts feature is behind a feature flag: `group_managed_accounts`. The flag is disabled by default. -To activate the feature, ask a GitLab administrator with Rails console access to run: - -```ruby -Feature.enable(:group_managed_accounts) -``` - -##### Credentials inventory for Group-managed accounts **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38133) in GitLab 12.8. - -Owners who manage user accounts in a group can view the following details of personal access tokens and SSH keys: - -- Owners -- Scopes -- Usage patterns - -To access the Credentials inventory of a group, navigate to **{shield}** **Security & Compliance > Credentials** in your group's sidebar. - -This feature is similar to the [Credentials inventory for self-managed instances](../../admin_area/credentials_inventory.md). - -##### Limiting lifetime of personal access tokens of users in Group-managed accounts **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118893) in GitLab 12.10. - -Users in a group managed account can optionally specify an expiration date for -[personal access tokens](../../profile/personal_access_tokens.md). -This expiration date is not a requirement, and can be set to any arbitrary date. - -Since personal access tokens are the only token needed for programmatic access to GitLab, organizations with security requirements may want to enforce more protection to require regular rotation of these tokens. - -###### Setting a limit - -Only a GitLab administrator or an owner of a Group-managed account can set a limit. Leaving it empty means that the [instance level restrictions](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only) on the lifetime of personal access tokens will apply. - -To set a limit on how long personal access tokens are valid for users in a group managed account: - -1. Navigate to the **{settings}** **Settings > General** page in your group's sidebar. -1. Expand the **Permissions, LFS, 2FA** section. -1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. -1. Click **Save changes**. - -Once a lifetime for personal access tokens is set, GitLab will: - -- Apply the lifetime for new personal access tokens, and require users managed by the group to set an expiration date that is no later than the allowed lifetime. -- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the allowed lifetime. Three hours is given to allow administrators/group owner to change the allowed lifetime, or remove it, before revocation takes place. - -##### Outer forks restriction for Group-managed accounts - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34648) in GitLab 12.9. - -Groups with group-managed accounts can disallow forking of projects to destinations outside the group. -To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. -When enabled, projects within the group can only be forked to other destinations within the group (including its subgroups). - -##### Other restrictions for Group-managed accounts - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12420) in GitLab 12.9. - -Projects within groups with enabled group-managed accounts are not to be shared with: - -- Groups outside of the parent group. -- Members who are not users managed by this group. - -This restriction also applies to projects forked from or to those groups. - -#### Assertions - -When using group-managed accounts, the following user details need to be passed to GitLab as SAML -assertions to be able to create a user. - -| Field | Supported keys | -|-----------------|----------------| -| Email (required)| `email`, `mail` | -| Full Name | `name` | -| First Name | `first_name`, `firstname`, `firstName` | -| Last Name | `last_name`, `lastname`, `lastName` | - ### Metadata configuration GitLab provides metadata XML that can be used to configure your Identity Provider. @@ -185,7 +61,7 @@ GitLab provides metadata XML that can be used to configure your Identity Provide Once you've set up your identity provider to work with GitLab, you'll need to configure GitLab to use it for authentication: 1. Navigate to the group's **Settings > SAML SSO**. -1. Find the SSO URL from your Identity Provider and enter it the **Identity provider single sign on URL** field. +1. Find the SSO URL from your Identity Provider and enter it the **Identity provider single sign-on URL** field. 1. Find and enter the fingerprint for the SAML token signing certificate in the **Certificate** field. 1. Click the **Enable SAML authentication for this group** toggle switch. 1. Click the **Save changes** button. @@ -193,42 +69,27 @@ Once you've set up your identity provider to work with GitLab, you'll need to co  NOTE: **Note:** -Please note that the certificate [fingerprint algorithm](#additional-setup-options) must be in SHA1. When configuring the identity provider, use a secure [signature algorithm](#additional-setup-options). - -## User access and management - -Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, please see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup). - -When a user tries to sign in with Group SSO, they'll need an account that's configured with one of the following: +Please note that the certificate [fingerprint algorithm](#additional-providers-and-setup-options) must be in SHA1. When configuring the identity provider, use a secure signature algorithm. -- [SCIM](scim_setup.md). -- [Group-managed accounts](#group-managed-accounts). -- A GitLab.com account. - -1. Click on the GitLab app in the identity provider's dashboard, or visit the Group's GitLab SSO URL. -1. Sign in to GitLab.com. The next time you connect on the same browser, you won't have to sign in again provided the active session has not expired. -1. Click on the **Authorize** button. - -On subsequent visits, users can access the group through the identify provider's dashboard or by visiting links directly. With the **enforce SSO** option turned on, users will be redirected to log in through the identity provider as required. - -### Role +### SSO enforcement -Upon first sign in, a new user is added to the parent group with the Guest role. Existing members with an appropriate role will have to elevate users to a higher role where relevant. +- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. +- [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. -If a user is already a member of the group, linking the SAML identity does not change their role. +With this option enabled, users must go through your group's GitLab single sign-on URL. They may also be added via SCIM, if configured. Users cannot be added manually, and may only access project/group resources via the UI by signing in through the SSO URL. -### Blocking access +However, users will not be prompted to sign in through SSO on each visit. GitLab will check whether a user has authenticated through SSO, and will only prompt the user to sign in via SSO if the session has expired. -To rescind access to the group: +We intend to add a similar SSO requirement for [Git and API activity](https://gitlab.com/gitlab-org/gitlab/-/issues/9152). -1. Remove the user from the identity provider or users list for the specific app. -1. Remove the user from the GitLab.com group. +When SSO enforcement is enabled for a group, users cannot share a project in the group outside the top-level group, even if the project is forked. -Even when **enforce SSO** is active, we recommend removing the user from the group. Otherwise, the user can sign in through the identity provider if they do not have an active session. +To disallow users to contribute outside of the top-level group, please see [Group Managed Accounts](group_managed_accounts.md). ## Providers -NOTE: **Note:** GitLab is unable to provide support for IdPs that are not listed here. +NOTE: **Note:** +GitLab is unable to provide support for IdPs that are not listed here. | Provider | Documentation | |----------|---------------| @@ -248,7 +109,8 @@ For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azu |--------------|----------------| | Identifier | Identifier (Entity ID) | | Assertion consumer service URL | Reply URL (Assertion Consumer Service URL) | -| Identity provider single sign on URL | Sign on URL | +| GitLab single sign-on URL | Sign on URL | +| Identity provider single sign-on URL | Login URL | | Certificate fingerprint | Thumbprint | We recommend: @@ -256,8 +118,6 @@ We recommend: - **Unique User Identifier (Name identifier)** set to `user.objectID`. - **nameid-format** set to persistent. -Set other user attributes and claims according to the [assertions table](#assertions). - ### Okta setup notes <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> @@ -266,19 +126,17 @@ For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & S | GitLab Setting | Okta Field | |--------------|----------------| | Identifier | Audience URI | -| Assertion consumer service URL | Single sign on URL | - -Under Okta's **Single sign on URL** field, check the option **Use this for Recipient URL and Destination URL**. +| Assertion consumer service URL | Single sign-on URL | +| GitLab single sign-on URL | Login page URL (under **Application Login Page** settings) | +| Identity provider single sign-on URL | Identity Provider Single Sign-On URL | -Please note that Okta's generic SAML app does not have a **Login URL** field, where the **Identity provider single sign on URL** would normally go. The **Identity provider single sign on URL** may be required the first time a user is logging in if they are having any difficulties. +Under Okta's **Single sign-on URL** field, check the option **Use this for Recipient URL and Destination URL**. We recommend: - **Application username** (NameID) set to **Custom** `user.getInternalProperty("id")`. - **Name ID Format** set to **Persistent**. -Set attribute statements according to the [assertions table](#assertions). - ### OneLogin setup notes The GitLab app listed in the OneLogin app catalog is for self-managed GitLab instances. @@ -290,15 +148,24 @@ For GitLab.com, use a generic SAML Test Connector such as the SAML Test Connecto | Assertion consumer service URL | Recipient | | Assertion consumer service URL | ACS (Consumer) URL | | Assertion consumer service URL (escaped version) | ACS (Consumer) URL Validator | -| GitLab single sign on URL | Login URL | +| GitLab single sign-on URL | Login URL | +| Identity provider single sign-on URL | SAML 2.0 Endpoint | Recommended `NameID` value: `OneLogin ID`. -Set parameters according to the [assertions table](#assertions). +### Additional providers and setup options + +The SAML standard means that a wide range of identity providers will work with GitLab. Unfortunately we have not verified connections with all SAML providers. +For more information, see our [discussion on providers](#providers). -### Additional setup options +Your identity provider may have relevant documentation. It may be generic SAML documentation, or specifically targeted for GitLab. Examples: + +- [Auth0](https://auth0.com/docs/protocols/saml/saml-idp-generic) +- [G Suite](https://support.google.com/a/answer/6087519?hl=en) +- [JumpCloud](https://support.jumpcloud.com/support/s/article/single-sign-on-sso-with-gitlab-2019-08-21-10-36-47) +- [PingOne by Ping Identity](https://docs.pingidentity.com/bundle/pingone/page/xsh1564020480660-1.html) -GitLab [isn't limited to the SAML providers listed above](#my-identity-provider-isnt-listed) but your Identity Provider may require additional configuration, such as the following: +Your Identity Provider may require additional configuration, such as the following: | Field | Value | Notes | |-------|-------|-------| @@ -319,24 +186,48 @@ GitLab [isn't limited to the SAML providers listed above](#my-identity-provider- If the information you need isn't listed above you may wish to check our [troubleshooting docs below](#i-need-additional-information-to-configure-my-identity-provider). -## Linking SAML to your existing GitLab.com account +## User access and management + +Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, please see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup). + +When a user tries to sign in with Group SSO, they will need an account that's configured with one of the following: + +- [SCIM](scim_setup.md). +- [Group-managed accounts](group_managed_accounts.md). +- A GitLab.com account. + +### Linking SAML to your existing GitLab.com account To link SAML to your existing GitLab.com account: 1. Sign in to your GitLab.com account. -1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. -1. Visit the SSO URL and click **Authorize**. +1. Locate and visit the **GitLab single sign-on URL** for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. If the sign-in URL is configured, users can connect to the GitLab app from the Identity Provider. +1. Click **Authorize**. 1. Enter your credentials on the Identity Provider if prompted. 1. You will be redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. -## Signing in to GitLab.com with SAML +On subsequent visits, you should be able to go [sign in to GitLab.com with SAML](#signing-in-to-gitlabcom-with-saml) or by visiting links directly. If the **enforce SSO** option is turned on, you will be redirected to sign in through the identity provider. -1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on a group's **Settings > SAML SSO** page. If configured, it might also be possible to sign in to GitLab starting from your Identity Provider. -1. Visit the SSO URL and click the **Sign in with Single Sign-On** button. -1. Enter your credentials on the Identity Provider if prompted. +### Signing in to GitLab.com with SAML + +1. Sign in to your identity provider. +1. From the list of apps, click on the "GitLab.com" app (The name is set by the administrator of the identity provider). 1. You will be signed in to GitLab.com and redirected to the group. -## Unlinking accounts +### Role + +The first time you sign in, GitLab adds you to the top-level parent group with the Guest role. Existing members with appropriate privileges can promote that new user. + +If a user is already a member of the group, linking the SAML identity does not change their role. + +### Blocking access + +To rescind access to the group, perform the following steps, in order: + +1. Remove the user from the user datastore on the identity provider or the list of users on the specific app. +1. Remove the user from the GitLab.com group. + +### Unlinking accounts Users can unlink SAML for a group from their profile page. This can be helpful if: @@ -359,7 +250,7 @@ For example, to unlink the `MyOrg` account, the following **Disconnect** button | Issuer | How GitLab identifies itself to the identity provider. Also known as a "Relying party trust identifier". | | Certificate fingerprint | Used to confirm that communications over SAML are secure by checking that the server is signing communications with the correct certificate. Also known as a certificate thumbprint. | -## Configuring on a self-managed GitLab instance +## Configuring on a self-managed GitLab instance **(PREMIUM ONLY)** For self-managed GitLab instances we strongly recommend using the [instance-wide SAML OmniAuth Provider](../../../integration/saml.md) instead. @@ -377,7 +268,7 @@ Group SAML on a self-managed instance is limited when compared to the recommende [instance-wide SAML](../../../integration/saml.md). The recommended solution allows you to take advantage of: - [LDAP compatibility](../../../administration/auth/ldap/index.md). -- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap) +- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap). - [Required groups](../../../integration/saml.md#required-groups-starter-only). - [Admin groups](../../../integration/saml.md#admin-groups-starter-only). - [Auditor groups](../../../integration/saml.md#auditor-groups-starter-only). @@ -449,7 +340,6 @@ Here are possible causes and solutions: | Cause | Solution | |------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | You've tried to link multiple SAML identities to the same user, for a given Identity Provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](#unlinking-accounts) from this GitLab account before attempting to sign in again. | -| The Identity Provider might be returning an inconsistent [NameID](#nameid). | Ask an admin of your Identity Provider to use the [SCIM API](../../../api/scim.md) to update your `extern_uid` to match the current **NameID**. | ### Message: "SAML authentication failed: Email has already been taken" @@ -463,6 +353,16 @@ Getting both of these errors at the same time suggests the NameID capitalization This can be prevented by configuring the [NameID](#nameid) to return a consistent value. Fixing this for an individual user involves [unlinking SAML in the GitLab account](#unlinking-accounts), although this will cause group membership and Todos to be lost. +### Message: "Request to link SAML account must be authorized" + +Ensure that the user who is trying to link their GitLab account has been added as a user within the identity provider's SAML app. + +### Stuck in a login "loop" + +Ensure that the **GitLab single sign-on URL** has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. + +Alternatively, when users need to [link SAML to their existing GitLab.com account](#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. + ### The NameID has changed | Cause | Solution | @@ -473,17 +373,6 @@ This can be prevented by configuring the [NameID](#nameid) to return a consisten Users will need to [unlink the current SAML identity](#unlinking-accounts) and [link their identity](#user-access-and-management) to the new SAML app. -### My identity provider isn't listed - -Not a problem, the SAML standard means that a wide range of identity providers will work with GitLab. Unfortunately we aren't familiar with all of them so can only offer support configuring the [listed providers](#providers). - -Your identity provider may also have relevant documentation. It may be generic SAML documentation, or specifically targeted for GitLab. Examples: - -- [Auth0](https://auth0.com/docs/protocols/saml/saml-idp-generic) -- [G Suite](https://support.google.com/a/answer/6087519?hl=en) -- [JumpCloud](https://support.jumpcloud.com/support/s/article/single-sign-on-sso-with-gitlab-2019-08-21-10-36-47) -- [PingOne by Ping Identity](https://docs.pingidentity.com/bundle/pingone/page/xsh1564020480660-1.html) - ### I need additional information to configure my identity provider Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index a891962b38e..13e9d623e2c 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -92,7 +92,8 @@ You can then test the connection by clicking on **Test Connection**. If the conn 1. Save your changes. For reference, you can view [an example configuration in the troubleshooting reference](../../../administration/troubleshooting/group_saml_scim.md#azure-active-directory). - NOTE: **Note:** If you used a unique identifier **other than** `objectId`, be sure to map it to `externalId`. + NOTE: **Note:** + If you used a unique identifier **other than** `objectId`, be sure to map it to `externalId`. 1. Below the mapping list click on **Show advanced options > Edit attribute list for AppName**. @@ -127,7 +128,8 @@ Before proceeding, be sure to complete the [GitLab configuration](#gitlab-config 1. If you see an **Admin** button in the top right, click the button. This will ensure you are in the Admin area. - TIP: **Tip:** If you're using the Developer Console, click **Developer Console** in the top + TIP: **Tip:** + If you're using the Developer Console, click **Developer Console** in the top bar and select **Classic UI**. Otherwise, you may not see the buttons described in the following steps: @@ -163,7 +165,7 @@ As long as [Group SAML](index.md) has been configured, prior to turning on sync, - By following these steps: 1. Sign in to GitLab.com if needed. - 1. Click on the GitLab app in the identity provider's dashboard or visit the **GitLab single sign on URL**. + 1. Click on the GitLab app in the identity provider's dashboard or visit the **GitLab single sign-on URL**. 1. Click on the **Authorize** button. New users and existing users on subsequent visits can access the group through the identify provider's dashboard or by visiting links directly. @@ -175,7 +177,7 @@ For role information, please see the [Group SAML page](index.md#user-access-and- To rescind access to the group, we recommend removing the user from the identity provider or users list for the specific app. -Upon the next sync, the user will be deprovisioned, which means that the user will be removed from the group. The user account will not be deleted unless using [group managed accounts](index.md#group-managed-accounts). +Upon the next sync, the user will be deprovisioned, which means that the user will be removed from the group. The user account will not be deleted unless using [group managed accounts](group_managed_accounts.md). ## Troubleshooting diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 842e2082be4..a370c9cf136 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -19,10 +19,13 @@ By using subgroups you can do the following: - **Make it easier to manage people and control visibility.** Give people different [permissions](../../permissions.md#group-members-permissions) depending on their group [membership](#membership). +For more information on allowed permissions in groups and projects, see +[visibility levels](../../../development/permissions.md#general-permissions). + ## Overview A group can have many subgroups inside it, and at the same time a group can have -only 1 parent group. It resembles a directory behavior or a nested items list: +only one immediate parent group. It resembles a directory behavior or a nested items list: - Group 1 - Group 1.1 @@ -86,7 +89,7 @@ of words that are not allowed to be used as group names see the [reserved names](../../reserved_names.md). Users can always create subgroups if they are explicitly added as an Owner (or -Maintainer, if that setting is enabled) to a parent group, even if group +Maintainer, if that setting is enabled) to an immediate parent group, even if group creation is disabled by an administrator in their settings. To create a subgroup: @@ -96,9 +99,9 @@ To create a subgroup:  -1. Create a new group like you would normally do. Notice that the parent group +1. Create a new group like you would normally do. Notice that the immediate parent group namespace is fixed under **Group path**. The visibility level can differ from - the parent group. + the immediate parent group.  @@ -110,12 +113,13 @@ Follow the same process to create any subsequent groups. ## Membership When you add a member to a subgroup, they inherit the membership and permission -level from the parent group. This model allows access to nested groups if you +level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. -Jobs for pipelines in subgroups can use [Runners](../../../ci/runners/README.md) registered to the parent group. This means secrets configured for the parent group are available to subgroup jobs. +Jobs for pipelines in subgroups can use [Runners](../../../ci/runners/README.md) registered to the parent group(s). +This means secrets configured for the parent group are available to subgroup jobs. -In addition, maintainers of projects that belong to subgroups can see the details of Runners registered to parent groups. +In addition, maintainers of projects that belong to subgroups can see the details of Runners registered to parent group(s). The group permissions for a member can be changed only by Owners, and only on the **Members** page of the group the member was added. diff --git a/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png b/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png Binary files differnew file mode 100644 index 00000000000..9277b013676 --- /dev/null +++ b/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png diff --git a/doc/user/incident_management/index.md b/doc/user/incident_management/index.md index 48805bda909..996f423e770 100644 --- a/doc/user/incident_management/index.md +++ b/doc/user/incident_management/index.md @@ -25,7 +25,7 @@ to create issues when alerts are triggered: checkbox to create an issue based on your own [issue templates](../project/description_templates.md#creating-issue-templates). For more information, see - [Taking Action on Incidents](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) **(ULTIMATE)**. + [Trigger actions from alerts](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) **(ULTIMATE)**. 1. To create issues from alerts, select the template in the **Issue Template** select box. 1. To send [separate email notifications](#notify-developers-of-alerts) to users @@ -34,9 +34,9 @@ to create issues when alerts are triggered: 1. Click **Save changes**. Appropriately configured alerts include an -[embedded chart](../project/integrations/prometheus.md#embedding-metrics-based-on-alerts-in-incident-issues) +[embedded chart](../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues) for the query corresponding to the alert. You can also configure GitLab to -[close issues](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) +[close issues](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) when you receive notification that the alert is resolved. ### Notify developers of alerts @@ -49,12 +49,35 @@ These emails contain details of the alert, and a link for more information. To send separate email notifications to users with [Developer permissions](../permissions.md), see [Configure incidents](#configure-incidents-ultimate). +## Configure PagerDuty integration + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/119018) in GitLab 13.2. + +You can set up a webhook with PagerDuty to automatically create a GitLab issue +for each PagerDuty incident. This configuration requires you to make changes +in both PagerDuty and GitLab: + +1. Sign in as a user with Maintainer [permissions](../permissions.md). +1. Navigate to **{settings}** **Settings > Operations > Incidents** and expand **Incidents**. +1. Select the **PagerDuty integration** tab: + +  + +1. Activate the integration, and save the changes in GitLab. +1. Copy the value of **Webhook URL**, as you'll need it in a later step. +1. Follow the steps described in the + [PagerDuty documentation](https://support.pagerduty.com/docs/webhooks) + to add the webhook URL to a PagerDuty webhook integration. + +To confirm the integration is successful, trigger a test incident from PagerDuty to +confirm that a GitLab issue is created from the incident. + ## Configure Prometheus alerts You can set up Prometheus alerts in: -- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics) installations. -- [Self-managed Prometheus](../project/integrations/prometheus.md#external-prometheus-instances) installations. +- [GitLab-managed Prometheus](../../operations/metrics/alerts.md) installations. +- [Self-managed Prometheus](../../operations/metrics/alerts.md#external-prometheus-instances) installations. Prometheus alerts are created by the special Alert Bot user. You can't remove this user, but it does not count toward your license limit. @@ -71,11 +94,11 @@ You can embed metrics anywhere [GitLab Markdown](../markdown.md) is used, such a comments on issues, and merge requests. Embedding metrics helps you share them when discussing incidents or performance issues. You can output the dashboard directly into any issue, merge request, epic, or any other Markdown text field in GitLab -by [copying and pasting the link to the metrics dashboard](../project/integrations/prometheus.md#embedding-gitlab-managed-kubernetes-metrics). +by [copying and pasting the link to the metrics dashboard](../../operations/metrics/embed.md#embedding-gitlab-managed-kubernetes-metrics). You can embed both -[GitLab-hosted metrics](../project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) and -[Grafana metrics](../project/integrations/prometheus.md#embedding-grafana-charts) +[GitLab-hosted metrics](../../operations/metrics/embed.md) and +[Grafana metrics](../../operations/metrics/embed_grafana.md) in incidents and issue templates. ### Context menu @@ -86,7 +109,7 @@ above the upper right corner of the panel. The options are: - [View logs](#view-logs-from-metrics-panel). - **Download CSV** - Data from embedded charts can be - [downloaded as CSV](../project/integrations/prometheus.md#downloading-data-as-csv). + [downloaded as CSV](../../operations/metrics/dashboards/index.md#downloading-data-as-csv). #### View logs from metrics panel @@ -94,7 +117,7 @@ above the upper right corner of the panel. The options are: > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) to [GitLab Core](https://about.gitlab.com/pricing/) 12.9. Viewing logs from a metrics panel can be useful if you're triaging an application -incident and need to [explore logs](../project/integrations/prometheus.md#view-logs-ultimate) +incident and need to [explore logs](../../operations/metrics/dashboards/index.md#view-logs-ultimate) from across your application. These logs help you understand what is affecting your application's performance and resolve any problems. diff --git a/doc/user/index.md b/doc/user/index.md index cc521c2a767..f50b712e2c3 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -46,10 +46,10 @@ GitLab is a Git-based platform that integrates a great number of essential tools - Deploying personal and professional static websites with [GitLab Pages](project/pages/index.md). - Integrating with Docker by using [GitLab Container Registry](packages/container_registry/index.md). - Tracking the development lifecycle by using [GitLab Value Stream Analytics](project/cycle_analytics.md). +- Provide support with [Service Desk](project/service_desk.md). With GitLab Enterprise Edition, you can also: -- Provide support with [Service Desk](project/service_desk.md). - Improve collaboration with: - [Merge Request Approvals](project/merge_requests/merge_request_approvals.md). **(STARTER)** - [Multiple Assignees for Issues](project/issues/multiple_assignees_for_issues.md). **(STARTER)** @@ -148,7 +148,7 @@ requests you're assigned to. [Snippets](snippets.md) are code blocks that you want to store in GitLab, from which you have quick access to. You can also gather feedback on them through -[Discussions](#Discussions). +[Discussions](#discussions). ## Keyboard shortcuts diff --git a/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png b/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png Binary files differnew file mode 100644 index 00000000000..564835a5dbe --- /dev/null +++ b/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index c17d1831b50..e24e669d994 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -6,8 +6,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Infrastructure as code with Terraform and GitLab +## Motivation + +The Terraform integration features within GitLab enable your GitOps / Infrastructure-as-Code (IaC) +workflows to tie into GitLab's authentication and authorization. These features focus on +lowering the barrier to entry for teams to adopt Terraform, collaborate effectively within +GitLab, and support Terraform best practices. + ## GitLab managed Terraform State +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. + [Terraform remote backends](https://www.terraform.io/docs/backends/index.html) enable you to store the state file in a remote, shared store. GitLab uses the [Terraform HTTP backend](https://www.terraform.io/docs/backends/types/http.html) @@ -27,6 +36,14 @@ To get started with a GitLab-managed Terraform State, there are two different op - [Use a local machine](#get-started-using-local-development). - [Use GitLab CI](#get-started-using-gitlab-ci). +## Permissions for using Terraform + +In GitLab version 13.1, [Maintainer access](../permissions.md) was required to use a +GitLab managed Terraform state backend. In GitLab versions 13.2 and greater, +[Maintainer access](../permissions.md) is required to lock, unlock and write to the state +(using `terraform apply`), while [Developer access](../permissions.md) is required to read +the state (using `terraform plan -lock=false`). + ## Get started using local development If you plan to only run `terraform plan` and `terraform apply` commands from your @@ -45,8 +62,7 @@ local machine, this is a simple way to get started: ``` 1. Create a [Personal Access Token](../profile/personal_access_tokens.md) with - the `api` scope. The Terraform backend is restricted to users with - [Maintainer access](../permissions.md) to the repository. + the `api` scope. 1. On your local machine, run `terraform init`, passing in the following options, replacing `<YOUR-PROJECT-NAME>`, `<YOUR-PROJECT-ID>`, `<YOUR-USERNAME>` and @@ -80,10 +96,6 @@ Next, [configure the backend](#configure-the-backend). After executing the `terraform init` command, you must configure the Terraform backend and the CI YAML file: -CAUTION: **Important:** -The Terraform backend is restricted to users with [Maintainer access](../permissions.md) -to the repository. - 1. In your Terraform project, define the [HTTP backend](https://www.terraform.io/docs/backends/types/http.html) by adding the following code block in a `.tf` file (such as `backend.tf`) to define the remote backend: @@ -95,64 +107,75 @@ to the repository. } ``` -1. In the root directory of your project repository, configure a `.gitlab-ci.yaml` file. - This example uses a pre-built image: +1. In the root directory of your project repository, configure a + `.gitlab-ci.yaml` file. This example uses a pre-built image which includes a + `gitlab-terraform` helper. For supported Terraform versions, see the [GitLab + Terraform Images project](https://gitlab.com/gitlab-org/terraform-images). ```yaml - image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest ``` -1. In the `.gitlab-ci.yaml` file, define some environment variables to ease development. In this - example, `GITLAB_TF_ADDRESS` is the URL of the GitLab instance where this pipeline - runs, and `TF_ROOT` is the directory where the Terraform commands must be executed: +1. In the `.gitlab-ci.yaml` file, define some environment variables to ease + development. In this example, `TF_ROOT` is the directory where the Terraform + commands must be executed, `TF_ADDRESS` is the URL to the state on the GitLab + instance where this pipeline runs, and the final path segment in `TF_ADDRESS` + is the name of the Terraform state. Projects may have multiple states, and + this name is arbitrary, so in this example we will set it to the name of the + project, and we will ensure that the `.terraform` directory is cached between + jobs in the pipeline using a cache key based on the state name: ```yaml variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production + TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} cache: + key: ${CI_PROJECT_NAME} paths: - - .terraform + - ${TF_ROOT}/.terraform ``` -1. In a `before_script`, pass a `terraform init` call containing configuration parameters - corresponding to variables required by the - [HTTP backend](https://www.terraform.io/docs/backends/types/http.html): +1. In a `before_script`, change to your `TF_ROOT`: ```yaml before_script: - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=gitlab-ci-token" -backend-config="password=${CI_JOB_TOKEN}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" stages: + - prepare - validate - build - - test - deploy + init: + stage: prepare + script: + - gitlab-terraform init + validate: stage: validate script: - - terraform validate + - gitlab-terraform validate plan: stage: build script: - - terraform plan - - terraform show + - gitlab-terraform plan + - gitlab-terraform plan-json + artifacts: + name: plan + paths: + - ${TF_ROOT}/plan.cache + reports: + terraform: ${TF_ROOT}/plan.json apply: stage: deploy environment: name: production script: - - terraform apply + - gitlab-terraform apply dependencies: - plan when: manual @@ -160,8 +183,9 @@ to the repository. - master ``` -1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline runs - the `terraform init`, `terraform validate`, and `terraform plan` commands. +1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline + runs the `gitlab-terraform init`, `gitlab-terraform validate`, and + `gitlab-terraform plan` commands. The output from the above `terraform` commands should be viewable in the job logs. @@ -176,15 +200,18 @@ you can expose details from `terraform plan` runs directly into a merge request enabling you to see statistics about the resources that Terraform will create, modify, or destroy. -Let's explore how to configure a GitLab Terraform Report artifact: +Let's explore how to configure a GitLab Terraform Report artifact. You can +either use a pre-built image which includes a `gitlab-terraform` helper as +above, where `gitlab-terraform plan-json` outputs the required artifact, or you +can configure this manually as follows: 1. For simplicity, let's define a few reusable variables to allow us to refer to these files multiple times: ```yaml variables: - PLAN: plan.tfplan - PLAN_JSON: tfplan.json + PLAN: plan.cache + PLAN_JSON: plan.json ``` 1. Install `jq`, a @@ -198,6 +225,18 @@ Let's explore how to configure a GitLab Terraform Report artifact: - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" ``` + NOTE: **Note:** + In distributions that use Bash (for example, Ubuntu), `alias` statements are not + expanded in non-interactive mode. If your pipelines fail with the error + `convert_report: command not found`, alias expansion can be activated explicitly + by adding a `shopt` command to your script: + + ```yaml + before_script: + - shopt -s expand_aliases + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" + ``` + 1. Define a `script` that runs `terraform plan` and `terraform show`. These commands pipe the output and convert the relevant bits into a store variable `PLAN_JSON`. This JSON is used to create a @@ -212,18 +251,18 @@ Let's explore how to configure a GitLab Terraform Report artifact: - terraform plan -out=$PLAN - terraform show --json $PLAN | convert_report > $PLAN_JSON artifacts: - name: plan - paths: - - $PLAN reports: terraform: $PLAN_JSON ``` - For a full example, see [Example `.gitlab-ci.yaml` file](#example-gitlab-ciyaml-file). + For a full example using the pre-built image, see [Example `.gitlab-ci.yaml` + file](#example-gitlab-ciyaml-file). + + For an example displaying multiple reports, see [`.gitlab-ci.yaml` multiple reports file](#multiple-terraform-plan-reports). 1. Running the pipeline displays the widget in the merge request, like this: -  +  1. Clicking the **View Full Log** button in the widget takes you directly to the plan output present in the pipeline logs: @@ -233,64 +272,114 @@ Let's explore how to configure a GitLab Terraform Report artifact: ### Example `.gitlab-ci.yaml` file ```yaml -image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' +image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest -# Default output file for Terraform plan variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} - PLAN: plan.tfplan - PLAN_JSON: tfplan.json - TF_ROOT: ${CI_PROJECT_DIR} + TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production + TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} cache: + key: ${CI_PROJECT_NAME} paths: - - .terraform + - ${TF_ROOT}/.terraform before_script: - - apk --no-cache add jq - - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" stages: + - prepare - validate - build - deploy +init: + stage: prepare + script: + - gitlab-terraform init + validate: stage: validate script: - - terraform validate + - gitlab-terraform validate plan: stage: build script: - - terraform plan -out=$PLAN - - terraform show --json $PLAN | convert_report > $PLAN_JSON + - gitlab-terraform plan + - gitlab-terraform plan-json artifacts: name: plan paths: - - ${TF_ROOT}/plan.tfplan + - ${TF_ROOT}/plan.cache reports: - terraform: ${TF_ROOT}/tfplan.json + terraform: ${TF_ROOT}/plan.json -# Separate apply job for manual launching Terraform as it can be destructive -# action. apply: stage: deploy environment: name: production script: - - terraform apply -input=false $PLAN + - gitlab-terraform apply dependencies: - plan when: manual only: - master +``` + +### Multiple Terraform Plan reports + +Starting with 13.2, you can display mutiple reports on the Merge Request page. The reports will also display the `artifact: name:`. See example below for a suggested setup. + +```yaml +image: + name: registry.gitlab.com/gitlab-org/gitlab-build-images:terraform + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + +cache: + paths: + - .terraform +stages: + - build + +.terraform-plan-generation: + stage: build + variables: + PLAN: plan.tfplan + JSON_PLAN_FILE: tfplan.json + before_script: + - cd ${TERRAFORM_DIRECTORY} + - terraform --version + - terraform init + - apk --no-cache add jq + script: + - terraform validate + - terraform plan -out=${PLAN} + - terraform show --json ${PLAN} | jq -r '([.resource_changes[]?.change.actions?]|flatten)|{"create":(map(select(.=="create"))|length),"update":(map(select(.=="update"))|length),"delete":(map(select(.=="delete"))|length)}' > ${JSON_PLAN_FILE} + artifacts: + reports: + terraform: ${TERRAFORM_DIRECTORY}/${JSON_PLAN_FILE} + +review_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "review/" + # Review will not include an artifact name + +staging_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "staging/" + artifacts: + name: Staging + +production_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "production/" + artifacts: + name: Production ``` diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 0d028cfe77a..b2b3f0f925c 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -14,7 +14,8 @@ website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitl Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/markdown-guide/) for a complete Kramdown reference. -NOTE: **Note:** We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md). +NOTE: **Note:** +We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md). ## GitLab Flavored Markdown (GFM) @@ -54,7 +55,7 @@ repository that were written using some of the nuances of GitLab's RedCarpet ver of Markdown. Since CommonMark uses slightly stricter syntax, these documents might now appear a little differently since we have transitioned to CommonMark. -It's usually quite easy to fix. For example, numbered lists with nested lists may +For example, numbered lists with nested lists may render incorrectly: ```markdown @@ -63,8 +64,8 @@ render incorrectly: - milk ``` -Simply add a space to each nested item to align the `-` with the first character of -the top list item (`C` in this case): +To correct their rendering, add a space to each nested item to align the `-` with the first +character of the top list item (`C` in this case): ```markdown 1. Chocolate @@ -76,7 +77,8 @@ the top list item (`C` in this case): - dark - milk -NOTE: **Note:** We will flag any significant differences between Redcarpet and CommonMark +NOTE: **Note:** +We will flag any significant differences between Redcarpet and CommonMark Markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine @@ -92,7 +94,7 @@ to change. GitLab makes full use of the standard (CommonMark) formatting, but also includes additional functionality useful for GitLab users. -It makes use of [new Markdown features](#new-GFM-markdown-extensions), +It makes use of [new Markdown features](#new-gfm-markdown-extensions), not found in standard Markdown: - [Color "chips" written in HEX, RGB or HSL](#colors) @@ -241,7 +243,7 @@ Sometimes you want to :monkey: around a bit and add some :star2: to your :speech You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. -If you're new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. +If you're new to this, don't be :fearful:. You can join the emoji :family:. All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: ``` @@ -252,7 +254,7 @@ Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/ma You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px" style="display:inline;margin:0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px" style="display:inline;margin:0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px" style="display:inline;margin:0">. People will <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px" style="display:inline;margin:0"> you for that. -If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. +If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> @@ -261,7 +263,8 @@ when rendered within GitLab, may appear different depending on the OS and browse Most emoji are natively supported on macOS, Windows, iOS, Android, and will fall back on image-based emoji where there is no support. -NOTE: **Note:** On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) +NOTE: **Note:** +On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. Ubuntu 18.04 (like many modern Linux distributions) has this font installed by default. @@ -402,14 +405,15 @@ a^2+b^2=c^2 _Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -NOTE: **Note:** This also works for the Asciidoctor `:stem: latexmath`. For details see +NOTE: **Note:** +This also works for the Asciidoctor `:stem: latexmath`. For details see the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). ### Special GitLab references -GFM recognizes special GitLab related references. For example, you can easily reference +GFM recognizes special GitLab related references. For example, you can reference an issue, a commit, a team member, or even the whole team within a project. GFM will turn -that reference into a link so you can navigate between them easily. +that reference into a link so you can navigate between them. Additionally, GFM recognizes certain cross-project references and also has a shorthand version to reference other projects from the same namespace. @@ -502,8 +506,6 @@ Second section content.  ---- - ### Wiki-specific Markdown The following examples show how links inside wikis behave. @@ -581,7 +583,7 @@ This snippet links to `<wiki_root>/miscellaneous.md`: ### Embedding metrics in GitLab Flavored Markdown -Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab flavored Markdown](../user/project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. +Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab flavored Markdown](../operations/metrics/embed.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. ## Standard Markdown and extensions in GitLab @@ -591,7 +593,7 @@ If a functionality is extended, the new option will be listed as a sub-section. ### Blockquotes -Blockquotes are an easy way to highlight information, such as a side-note. It's generated +Blockquotes are useful to highlight information, such as a side-note. It's generated by starting the lines of the blockquote with `>`: ```markdown @@ -637,9 +639,9 @@ you can quote that without having to manually prepend `>` to every line! ### Code spans and blocks -You can easily highlight anything that should be viewed as code and not simple text. +You can highlight anything that should be viewed as code and not simple text. -Simple inline code is easily highlighted with single backticks `` ` ``: +Simple inline code is highlighted with single backticks `` ` ``: ```markdown Inline `code` has `back-ticks around` it. @@ -781,7 +783,8 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ -NOTE: **Note:** Strikethrough is not part of the core Markdown standard, but is part of GFM. +NOTE: **Note:** +Strikethrough is not part of the core Markdown standard, but is part of GFM. #### Multiple underscores in words and mid-word emphasis @@ -1150,7 +1153,7 @@ A line break will be inserted (a new paragraph will start) if the previous text ended with two newlines, like when you hit <kbd>Enter</kbd> twice in a row. If you only use one newline (hit <kbd>Enter</kbd> once), the next sentence will be part of the same paragraph. This is useful if you want to keep long lines from wrapping, and keep -them easily editable: +them editable: ```markdown Here's a line for us to start with. @@ -1248,7 +1251,8 @@ Do not change to reference style links. Some text to show that the reference links can follow later. -NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki +NOTE: **Note:** +Relative links do not allow the referencing of project files in a wiki page, or a wiki page in a project file. The reason for this is that a wiki is always in a separate Git repository in GitLab. For example, `[I'm a reference-style link](style)` will point the link to `wikis/style` only when the link is inside of a wiki Markdown file. @@ -1275,7 +1279,7 @@ GFM will auto-link almost any URL you put into your text: ### Lists -Ordered and unordered lists can be easily created. +Ordered and unordered lists can be created. For an ordered list, add the number you want the list to start with, like `1.`, followed by a space, at the start of each line for ordered lists. diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 8a7c70ec74d..542efba1fae 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Composer Repository **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15886) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15886) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. With the GitLab Composer Repository, every project can have its own space to store [Composer](https://getcomposer.org/) packages. @@ -19,7 +19,7 @@ This option is available only if your GitLab administrator has After the Composer Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 020028dfc10..41b420ce7f6 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -22,7 +22,7 @@ This option is available only if your GitLab administrator has After the Conan Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -85,7 +85,7 @@ Next, you will create a package for that recipe by running `conan create` provid conan create . my-org+my-group+my-project/beta ``` -NOTE: **Note** +NOTE: **Note:** Current [naming restrictions](#package-recipe-naming-convention) require you to name the `user` value as the `+` separated path of your project on GitLab. The example above would create a package belonging to this project: `https://gitlab.com/my-org/my-group/my-project` with a channel of `beta`. @@ -129,12 +129,12 @@ Once you have a personal access token and have [set your Conan remote](#adding-t conan user <gitlab_username or deploy_token_username> -r gitlab -p <personal_access_token or deploy_token> ``` -Note: **Note** +NOTE: **Note:** If you named your remote something other than `gitlab`, your remote name should be used in this command instead of `gitlab`. From now on, when you run commands using `--remote=gitlab`, your username and password will automatically be included in the requests. -Note: **Note** +NOTE: **Note:** The personal access token is not stored locally at any moment. Conan uses JWT, so when you run this command, Conan will request an expirable token from GitLab using your token. The JWT does expire on a regular basis, so you will need to re-enter your personal access token when that happens. Alternatively, you could explicitly include your credentials in any given command. @@ -152,7 +152,7 @@ If you'd like Conan to always use GitLab as the registry for your package, you c conan remote add_ref Hello/0.1@my-group+my-project/beta gitlab ``` -NOTE: **Note** +NOTE: **Note:** The package recipe does include the version, so setting the default remote for `Hello/0.1@user/channel` will not work for `Hello/0.2@user/channel`. This functionality is best suited for when you want to consume or install packages from the GitLab registry without having to specify a remote. diff --git a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png Binary files differdeleted file mode 100644 index 93c9e00a8f5..00000000000 --- a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png +++ /dev/null diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index eb1933de62a..429d29b7677 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -71,7 +71,7 @@ This view will: - Allow you to [delete](#delete-images-from-within-gitlab) one or more image repository. - Allow you to navigate to the image repository details page. - Show a **Quick start** dropdown with the most common commands to log in, build and push -- Optionally, a banner will be visible if the [expiration policy](#expiration-policy) is enabled for this project. +- Optionally, a banner will be visible if the [cleanup policy](#cleanup-policy) is enabled for this project. ### Control Container Registry for your group @@ -248,10 +248,10 @@ should look similar to this: ```yaml build: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY/group/project/image:latest . @@ -262,10 +262,10 @@ You can also make use of [other variables](../../../ci/variables/README.md) to a ```yaml build: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -288,9 +288,9 @@ when needed. Changes to `master` also get tagged as `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:19.03.11 +image: docker:19.03.12 services: - - docker:19.03.11-dind + - docker:19.03.12-dind stages: - build @@ -363,9 +363,9 @@ Below is an example of what your `.gitlab-ci.yml` should look like: ```yaml build: - image: $CI_REGISTRY/group/project/docker:19.03.11 + image: $CI_REGISTRY/group/project/docker:19.03.12 services: - - name: $CI_REGISTRY/group/project/docker:19.03.11-dind + - name: $CI_REGISTRY/group/project/docker:19.03.12-dind alias: docker stage: build script: @@ -373,7 +373,7 @@ Below is an example of what your `.gitlab-ci.yml` should look like: - docker run my-docker-image /script/to/run/tests ``` -If you forget to set the service alias, the `docker:19.03.11` image won't find the +If you forget to set the service alias, the `docker:19.03.12` image won't find the `dind` service, and an error like the following will be thrown: ```plaintext @@ -443,10 +443,10 @@ stages: - clean build_image: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -459,10 +459,10 @@ build_image: - master delete_image: - image: docker:19.03.11 + image: docker:19.03.12 stage: clean services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_PROJECT_PATH:$CI_COMMIT_REF_SLUG REG_SHA256: ade837fc5224acd8c34732bf54a94f579b47851cc6a7fd5899a98386b782e228 @@ -486,25 +486,33 @@ You can download the latest `reg` release from the code example by changing the `REG_SHA256` and `REG_VERSION` variables defined in the `delete_image` job. -### Delete images using an expiration policy +### Delete images by using a cleanup policy -You can create a per-project [expiration policy](#expiration-policy) to ensure -older tags and images are regularly removed from the Container Registry. +You can create a per-project [cleanup policy](#cleanup-policy) to ensure older tags and images are regularly removed from the +Container Registry. -## Expiration policy +## Cleanup policy -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15398) in GitLab 12.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15398) in GitLab 12.8. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/218737) from "expiration policy" to "cleanup policy" in GitLab 13.2. + +For a specific project, if you want to remove tags you no longer need, +you can create a cleanup policy. When the policy is applied, tags matching the regex pattern are removed. +The underlying layers and images remain. + +To delete the underlying layers and images no longer associated with any tags, Instance Administrators can use +[garbage collection](../../../administration/packages/container_registry.md#removing-unused-layers-not-referenced-by-manifests) with the `-m` switch. NOTE: **Note:** -For GitLab.com, expiration policies are not available for projects created before GitLab 12.8. -For self-managed instances, expiration policies may be enabled by an admin in the -[CI/CD Package Registry settings](./../../admin_area/settings/index.md#cicd). +For GitLab.com, cleanup policies are not available for projects created +before this feature was deployed to production (February 2020). +Support for pre-existing projects on GitLab.com +[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/196124). +For self-managed instances, cleanup policies may be enabled by an admin in the +[GitLab application settings](../../../api/settings.md#change-application-settings) by setting `container_expiration_policies_enable_historic_entries` to true. Note the inherent [risks involved](./index.md#use-with-external-container-registries). -It is possible to create a per-project expiration policy, so that you can make sure that -older tags and images are regularly removed from the Container Registry. - -The expiration policy algorithm starts by collecting all the tags for a given repository in a list, +The cleanup policy algorithm starts by collecting all the tags for a given repository in a list, then goes through a process of excluding tags from it until only the ones to be deleted remain: 1. Collect all the tags for a given repository in a list. @@ -513,43 +521,41 @@ then goes through a process of excluding tags from it until only the ones to be 1. Excludes any tags that do not have a manifest (not part of the options). 1. Orders the remaining tags by `created_date`. 1. Excludes from the list the N tags based on the `keep_n` value (Number of tags to retain). -1. Excludes from the list the tags more recent than the `older_than` value (Expiration interval). +1. Excludes from the list the tags more recent than the `older_than` value (Cleanup interval). 1. Excludes from the list any tags matching the `name_regex_keep` value (Images to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. -### Managing project expiration policy through the UI - -To manage project expiration policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag expiration policy**. +### Managing project cleanup policy through the UI - +To manage project cleanup policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag cleanup policy**. The UI allows you to configure the following: -- **Expiration policy:** enable or disable the expiration policy. -- **Expiration interval:** how long tags are exempt from being deleted. -- **Expiration schedule:** how often the cron job checking the tags should run. +- **Cleanup policy:** enable or disable the cleanup policy. +- **Cleanup interval:** how long tags are exempt from being deleted. +- **Cleanup schedule:** how often the cron job checking the tags should run. - **Number of tags to retain:** how many tags to _always_ keep for each image. -- **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be expired. To qualify all tags for expiration, use the default value of `.*`. +- **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be cleaned up. To qualify all tags for cleanup, use the default value of `.*`. - **Docker tags with names matching this regex pattern will be preserved:** the regex used to determine what tags should be preserved. To preserve all tags, use the default value of `.*`. -#### Troubleshooting expiration policies +#### Troubleshooting cleanup policies If you see the following message: -"Something went wrong while updating the expiration policy." +"Something went wrong while updating the cleanup policy." Check the regex patterns to ensure they are valid. You can use [Rubular](https://rubular.com/) to check your regex. View some common [regex pattern examples](#regex-pattern-examples). -### Managing project expiration policy through the API +### Managing project cleanup policy through the API -You can set, update, and disable the expiration policies using the GitLab API. +You can set, update, and disable the cleanup policies using the GitLab API. Examples: -- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: +- Select all tags, keep at least 1 tag per image, clean up any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: ```shell curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' 'https://gitlab.example.com/api/v4/projects/2' @@ -560,15 +566,15 @@ See the API documentation for further details: [Edit project](../../../api/proje ### Use with external container registries When using an [external container registry](./../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), -running an expiration policy on a project may have some performance risks. If a project is going to run +running a cleanup policy on a project may have some performance risks. If a project is going to run a policy that will remove large quantities of tags (in the thousands), the GitLab background jobs that -run the policy may get backed up or fail completely. It is recommended you only enable container expiration +run the policy may get backed up or fail completely. It is recommended you only enable container cleanup policies for projects that were created before GitLab 12.8 if you are confident the amount of tags being cleaned up will be minimal. ### Regex pattern examples -Expiration policies use regex patterns to determine which tags should be preserved or removed, both in the UI and the API. +Cleanup policies use regex patterns to determine which tags should be preserved or removed, both in the UI and the API. Here are examples of regex patterns you may want to use: @@ -596,6 +602,15 @@ Here are examples of regex patterns you may want to use: (?:v.+|master|release) ``` +## Use the Container Registry to store Helm Charts + +With the launch of [Helm v3](https://helm.sh/docs/topics/registries/), +you can use the Container Registry to store Helm Charts. However, due to the way metadata is passed +and stored by Docker, it is not possible for GitLab to parse this data and meet performance standards. +[This epic](https://gitlab.com/groups/gitlab-org/-/epics/2313) updates the architecture of the Container Registry to support Helm Charts. + +You can read more about the above challenges [here](https://gitlab.com/gitlab-org/gitlab/-/issues/38047#note_298842890). + ## Limitations - Moving or renaming existing Container Registry repositories is not supported @@ -603,7 +618,7 @@ once you have pushed images, because the images are signed, and the signature includes the repository name. To move or rename a repository with a Container Registry, you will have to delete all existing images. - Prior to GitLab 12.10, any tags that use the same image ID as the `latest` tag -will not be deleted by the expiration policy. +will not be deleted by the cleanup policy. ## Troubleshooting the GitLab Container Registry diff --git a/doc/user/packages/img/group_packages_list_v13_0.png b/doc/user/packages/img/group_packages_list_v13_0.png Binary files differdeleted file mode 100644 index 8cf3fd1a131..00000000000 --- a/doc/user/packages/img/group_packages_list_v13_0.png +++ /dev/null diff --git a/doc/user/packages/img/package_detail_v13_0.png b/doc/user/packages/img/package_detail_v13_0.png Binary files differdeleted file mode 100644 index dcfbc0a4787..00000000000 --- a/doc/user/packages/img/package_detail_v13_0.png +++ /dev/null diff --git a/doc/user/packages/img/project_packages_list_v13_0.png b/doc/user/packages/img/project_packages_list_v13_0.png Binary files differdeleted file mode 100644 index 468a6fe6467..00000000000 --- a/doc/user/packages/img/project_packages_list_v13_0.png +++ /dev/null diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 6e59a87ae36..ab9cdc204f8 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -6,11 +6,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Package Registry -GitLab Packages allows organizations to utilize GitLab as a private repository -for a variety of common package managers. Users are able to build and publish +With the GitLab Package Registry, you can use GitLab as a private or public repository +for a variety of common package managers. You can build and publish packages, which can be easily consumed as a dependency in downstream projects. -The Packages feature allows GitLab to act as a repository for the following: +GitLab acts as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | @@ -22,87 +22,88 @@ The Packages feature allows GitLab to act as a repository for the following: | [NuGet Repository](nuget_repository/index.md) **(PREMIUM)** | The GitLab NuGet Repository will enable every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.8+ | | [PyPi Repository](pypi_repository/index.md) **(PREMIUM)** | The GitLab PyPi Repository will enable every project in GitLab to have its own space to store [PyPi](https://pypi.org/) packages. | 12.10+ | | [Go Proxy](go_proxy/index.md) **(PREMIUM)** | The Go proxy for GitLab enables every project in GitLab to be fetched with the [Go proxy protocol](https://proxy.golang.org/). | 13.1+ | -| [Composer Repository](composer_repository/index.md) **(PREMIUM)** | The GitLab Composer Repository will enable every project in GitLab to have its own space to store [Composer](https://getcomposer.org/) packages. | 13.1+ | +| [Composer Repository](composer_repository/index.md) **(PREMIUM)** | The GitLab Composer Repository will enable every project in GitLab to have its own space to store [Composer](https://getcomposer.org/) packages. | 13.2+ | -## Enable the Package Registry for your project +## View packages -If you cannot find the **{package}** **Packages & Registries > Package -Registry** entry under your project's sidebar, ensure that: +You can view packages for your project or group. -1. The GitLab Package Registry has been enabled by your administrator (following - [this documentation](../../administration/packages/index.md)); and -1. The Package Registry has been enabled for your project. +1. Go to the project or group. +1. Go to **{package}** **Packages & Registries > Package Registry**. -Once an administrator has enabled the GitLab Package Registry for your GitLab -instance, to enable Package Registry for your project: +You can search, sort, and filter packages on this page. -1. Go to your project's **Settings > General** page. -1. Expand the **Visibility, project features, permissions** section and enable the -**Packages** feature on your project. -1. Press **Save changes** for the changes to take effect. You should now be able to -see the **Packages & Registries > Package Registry** link in the sidebar. +For information on how to create and upload a package, view the GitLab documentation for your package type. -### View Packages for your project +## Use GitLab CI/CD to build packages -Navigating to your project's **{package}** **Packages & Registries > Package Registry** will show a list -of all packages that have been added to your project. +You can use [GitLab CI/CD](./../../ci/README.md) to build packages. +For Maven and NPM packages, and Composer dependencies, you can +authenticate with GitLab by using the `CI_JOB_TOKEN`. - +CI/CD templates, which you can use to get started, are in [this repo](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). -On this page, you can: +Learn more about [using CI/CD to build Maven packages](maven_repository/index.md#creating-maven-packages-with-gitlab-cicd) +and [NPM packages](npm_registry/index.md#publishing-a-package-with-cicd). -- View all the packages that have been uploaded to the project. -- Sort the packages list by created date, version or name. -- Filter the list by package name. -- Change tabs to display packages of a certain type. -- Remove a package (if you have suitable [permissions](../permissions.md)). -- Navigate to specific package detail page. +If you use CI/CD to build a package, extended activity +information is displayed when you view the package details: -### View Packages for your group + -You can view all packages belonging to a group by navigating to **{package}** -**Packages & Registries > Package Registry** from the group sidebar. +You can view which pipeline published the package, as well as the commit and +user who triggered it. - +## Download a package -On this page, you can: +To download a package: -- View all the packages that have been uploaded to each of the groups projects. -- Sort the packages list by created date, version, name or project. -- Filter the list by package name. -- Change tabs to display packages of a certain type. -- Navigate to specific package detail page. +1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Click the name of the package you want to download. +1. In the **Activity** section, click the name of the package you want to download. -### View additional package information +## Delete a package -Additional package information can be viewed by browsing to the package details -page from the either the project or group list. +You cannot edit a package after you publish it in the Package Registry. Instead, you +must delete and recreate it. - +- You cannot delete packages from the group view. You must delete them from the project view instead. + See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/227714) for details. +- You must have suitable [permissions](../permissions.md). -On this page you can: +You can delete packages by using [the API](../../api/packages.md#delete-a-project-package) or the UI. -- See the extended package information, including metadata. This is unique to -each package type and will display different information for different types. -- View quick installation and registry setup instructions. These are a shortcut -for users who have already set up the Package Registry and just want quick -installation instructions. -- View the package activity, including when and how a package was published. -- View and download the contents of the package. Outside of installing a -package via a manager, you can also download the files individually. -- Delete the package (if you have suitable [permissions](../permissions.md)). +To delete a package in the UI: -### Build packages via GitLab CI/CD +1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Find the name of the package you want to delete. +1. Click **Delete**. -Some of the supported packages can be built via [GitLab CI/CD](./../../ci/README.md) -using the `CI_JOB_TOKEN`. If a package is built this way, then extended activity -information is displayed on the package details page: +The package is permanently deleted. - +## Disable the Package Registry -You can view which pipeline published the package, as well as the commit and -user who triggered it. To see if a package type supports being built via CI/CD, -check the specific documentation for your package type. +The Package Registry is automatically enabled. + +If you are using a self-managed instance of GitLab, your administrator can remove +the menu item, **{package}** **Packages & Registries**, from the GitLab sidebar. For more information, +see the [administration documentation](../../administration/packages/index.md). + +You can also remove the Package Registry for your project specifically: + +1. In your project, go to **{settings}** **Settings > General**. +1. Expand the **Visibility, project features, permissions** section and disable the + **Packages** feature. +1. Click **Save changes**. + +The **{package}** **Packages & Registries > Package Registry** entry is removed from the sidebar. + +## Package workflows + +Learn how to use the GitLab Package Registry to build your own custom package workflow. + +- [Use a project as a package registry](./workflows/project_registry.md) to publish all of your packages to one project. +- Publish multiple different packages from one [monorepo project](./workflows/monorepo.md). ## Suggested contributions @@ -125,10 +126,3 @@ are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_reques | [RubyGems](https://gitlab.com/gitlab-org/gitlab/-/issues/803) | Use GitLab to host your own gems. | | [SBT](https://gitlab.com/gitlab-org/gitlab/-/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | | [Vagrant](https://gitlab.com/gitlab-org/gitlab/-/issues/36899) | Securely host your Vagrant boxes in local repositories. | - -## Package workflows - -Learning how to use the GitLab Package Registry will help you build your own custom package workflow. - -- [Use a project as a package registry](./workflows/project_registry.md) to publish all of your packages to one project. -- [Working with a monorepo](./workflows/monorepo.md): Learn how to publish multiple different packages from one monorepo project. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 2d40a623fb8..b194b7d837a 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -23,7 +23,7 @@ After the Packages feature is enabled, the Maven Repository will be available fo all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -821,6 +821,16 @@ user's home location (in this case the user is `root` since it runs in a Docker container), and Maven will use the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). +### Version validation + +The version string is validated using the following regex. + +```ruby +\A(\.?[\w\+-]+\.?)+\z +``` + +You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/rrLQqUXjfKEoL6). + ## Troubleshooting ### Useful Maven command line options diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 4d60c227ccd..390b2c28e10 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -25,7 +25,7 @@ This option is available only if your GitLab administrator has After the NPM registry is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -238,7 +238,8 @@ The regex that is used for naming is validating all package names from all packa It allows for capital letters, while NPM does not, and allows for almost all of the characters NPM allows with a few exceptions (`~` is not allowed). -NOTE: **Note:** Capital letters are needed because the scope is required to be +NOTE: **Note:** +Capital letters are needed because the scope is required to be identical to the top level namespace of the project. So, for example, if your project path is `My-Group/project-foo`, your package must be named `@My-Group/any-package-name`. `@my-group/any-package-name` will not work. @@ -306,10 +307,12 @@ stages: deploy: stage: deploy script: - - echo '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken=${CI_JOB_TOKEN}'>.npmrc + - echo '//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}'>.npmrc - npm publish ``` +Learn more about [using `CI_JOB_TOKEN` to authenticate to the GitLab NPM registry](#authenticating-with-a-ci-job-token). + ## Troubleshooting ### Error running yarn with NPM registry diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 6ada68dcceb..4ee5e5c4627 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -63,7 +63,7 @@ This option is available only if your GitLab administrator has After the NuGet Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 6f6609d82b1..615741cc303 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -28,7 +28,7 @@ This option is available only if your GitLab administrator has After the PyPi Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 21435c41d68..ba62cf81847 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -25,7 +25,7 @@ To add or import a user, you can follow the ## Principles behind permissions -See our [product handbook on permissions](https://about.gitlab.com/handbook/product/#permissions-in-gitlab) +See our [product handbook on permissions](https://about.gitlab.com/handbook/product/gitlab-the-product/#permissions-in-gitlab). ## Instance-wide user permissions @@ -47,7 +47,7 @@ The following table depicts the various user permission levels in a project. |---------------------------------------------------|---------|------------|-------------|----------|--------| | Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | View Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | @@ -86,7 +86,7 @@ The following table depicts the various user permission levels in a project. | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create/edit requirements **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Pull [packages](packages/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Publish [packages](packages/index.md) **(PREMIUM)**| | | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) **(PREMIUM)**| | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | | Create/edit/delete [Releases](project/releases/index.md)| | | ✓ | ✓ | ✓ | | Create new branches | | | ✓ | ✓ | ✓ | @@ -120,6 +120,7 @@ The following table depicts the various user permission levels in a project. | Rewrite/remove Git tags | | | ✓ | ✓ | ✓ | | Manage Feature Flags **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | +| Run CI/CD pipeline against a protected branch | | | ✓ (*5*) | ✓ | ✓ | | Use environment terminals | | | | ✓ | ✓ | | Run Web IDE's Interactive Web Terminals **(ULTIMATE ONLY)** | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | @@ -139,7 +140,10 @@ The following table depicts the various user permission levels in a project. | Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | | Remove GitLab Pages | | | | ✓ | ✓ | | Manage clusters | | | | ✓ | ✓ | +| Manage Project Operations | | | | ✓ | ✓ | | View Pods logs | | | | ✓ | ✓ | +| Read Terraform state | | | ✓ | ✓ | ✓ | +| Manage Terraform state | | | | ✓ | ✓ | | Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | | Manage Error Tracking | | | | ✓ | ✓ | @@ -154,6 +158,7 @@ The following table depicts the various user permission levels in a project. | Remove project | | | | | ✓ | | Archive project | | | | | ✓ | | Delete issues | | | | | ✓ | +| Delete pipelines | | | | | ✓ | | Delete merge request | | | | | ✓ | | Disable notification emails | | | | | ✓ | | Force push to protected branches (*4*) | | | | | | @@ -246,6 +251,7 @@ group. | Create project in group | | | ✓ (3) | ✓ (3) | ✓ (3) | | Share (invite) groups with groups | | | | | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| Create/edit/delete iterations | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | @@ -267,8 +273,8 @@ group. | View Issues analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Billing **(FREE ONLY)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | -| View Usage Quotas **(FREE ONLY)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | +| View Billing **(FREE ONLY)** | | | | | ✓ (4) | +| View Usage Quotas **(FREE ONLY)** | | | | | ✓ (4) | 1. Groups can be set to [allow either Owners or Owners and Maintainers to create subgroups](group/subgroups/index.md#creating-a-subgroup) @@ -281,7 +287,7 @@ group. ### Subgroup permissions When you add a member to a subgroup, they inherit the membership and -permission level from the parent group. This model allows access to +permission level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. To learn more, read through the documentation on diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index 3c6f2989091..a70d11438f4 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -35,7 +35,8 @@ As an administrator, you can delete a user account by: - **Delete user and contributions** to delete the user and their associated records. -DANGER: **Danger:** Using the **Delete user and contributions** option may result +DANGER: **Danger:** +Using the **Delete user and contributions** option may result in removing more data than intended. Please see [associated records](#associated-records) below for additional details. diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index bfcaeaf6a15..4f769f9a671 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -5,9 +5,9 @@ group: Access info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Two-Factor Authentication +# Two-factor authentication -Two-factor Authentication (2FA) provides an additional level of security to your +Two-factor authentication (2FA) provides an additional level of security to your GitLab account. Once enabled, in addition to supplying your username and password to login, you'll be prompted for a code generated by your one time password authenticator. For example, a password manager on one of your devices. @@ -62,7 +62,7 @@ To enable 2FA: 1. Click **Submit**. If the pin you entered was correct, you'll see a message indicating that -Two-Factor Authentication has been enabled, and you'll be presented with a list +two-factor authentication has been enabled, and you'll be presented with a list of [recovery codes](#recovery-codes). Make sure you download them and keep them in a safe place. diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 663a2888ee7..7a871afd861 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -22,7 +22,7 @@ See the [authentication topic](../../topics/authentication/index.md) for more de ### Unknown sign-in -GitLab will notify you if a sign-in occurs that is from an unknown IP address. +GitLab will notify you if a sign-in occurs that is from an unknown IP address or device. See [Unknown Sign-In Notification](unknown_sign_in_notification.md) for more details. ## User profile diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index ee228050945..dbf486e399e 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -187,7 +187,7 @@ To minimize the number of notifications that do not require any action, from [Gi | Remove milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected | | New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher | | Failed pipeline | The author of the pipeline | -| Fixed pipeline | The author of the pipeline. Disabled by default. To activate it you must [enable the `ci_pipeline_fixed_notifications` feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-in-development). | +| Fixed pipeline | The author of the pipeline. Enabled by default. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24309) in GitLab 13.1. Administrators can disable this notification option using the `ci_pipeline_fixed_notifications` [feature flag](../../administration/feature_flags.md). | | Successful pipeline | The author of the pipeline, if they have the custom notification setting for successful pipelines set. If the pipeline failed previously, a `Fixed pipeline` message will be sent for the first successful pipeline after the failure, then a `Successful pipeline` message for any further successful pipelines. | | New epic **(ULTIMATE)** | | | Close epic **(ULTIMATE)** | | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index e2c3dc74cf1..59ca124f566 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -19,6 +19,7 @@ Personal access tokens expire on the date you define, at midnight UTC. - GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that will expire in under seven days. The owners of these tokens are notified by email. - In GitLab Ultimate, administrators may [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only). +- In GitLab Ultimate, administrators may [toggle enforcement of personal access token expiry](../admin_area/settings/account_and_limit_settings.md#optional-enforcement-of-personal-access-token-expiry-ultimate-only). For examples of how you can use a personal access token to authenticate with the API, see the following section from our [API Docs](../../api/README.md#personalproject-access-tokens). @@ -43,6 +44,10 @@ profile. At any time, you can revoke any personal access token by clicking the respective **Revoke** button under the **Active Personal Access Token** area. +### Token activity + +You can see when a token was last used from the **Personal Access Tokens** page. Updates to the token usage is fixed at once per 24 hours. Requests to [API resources](../../api/api_resources.md) and the [GraphQL API](../../api/graphql/index.md) will update a token's usage. + ## Limiting scopes of a personal access token Personal access tokens can be created with one or more scopes that allow various diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index a5fa3cf373f..b94ae958d3b 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -63,7 +63,10 @@ Dark theme currently only works with the 'Dark' syntax highlighting. NOTE: **Note:** GitLab uses the [rouge Ruby library](http://rouge.jneen.net/ "Rouge website") -for syntax highlighting. For a list of supported languages visit the rouge website. +for syntax highlighting outside of any Editor context. The WebIDE (like Snippets) +uses [Monaco Editor](https://microsoft.github.io/monaco-editor/) and it's provided [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for +syntax highlighting. For a list of supported languages, visit the documentation of +the respective libraries. Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. diff --git a/doc/user/profile/unknown_sign_in_notification.md b/doc/user/profile/unknown_sign_in_notification.md index 200358bb050..6a6820bb2d4 100644 --- a/doc/user/profile/unknown_sign_in_notification.md +++ b/doc/user/profile/unknown_sign_in_notification.md @@ -9,16 +9,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27211) in GitLab 13.0. -When a user successfully signs in from a previously unknown IP address, +NOTE: **Note:** +This feature is enabled by default for self-managed instances. Administrators may disable this feature +through the [Sign-in restrictions](../admin_area/settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins) section of the UI. +The feature is always enabled on GitLab.com. + +When a user successfully signs in from a previously unknown IP address or device, GitLab notifies the user by email. In this way, GitLab proactively alerts users of potentially malicious or unauthorized sign-ins. -There are two methods used to identify a known sign-in: +There are several methods used to identify a known sign-in. All methods must fail +for a notification email to be sent. - Last sign-in IP: The current sign-in IP address is checked against the last sign-in IP address. - Current active sessions: If the user has an existing active session from the same IP address. See [Active Sessions](active_sessions.md). +- Cookie: After successful sign in, an encrypted cookie is stored in the browser. + This cookie is set to expire 14 days after the last successful sign in. ## Example email diff --git a/doc/user/project/bulk_editing.md b/doc/user/project/bulk_editing.md index 6d091c431a2..c4a6aea807c 100644 --- a/doc/user/project/bulk_editing.md +++ b/doc/user/project/bulk_editing.md @@ -14,7 +14,7 @@ For more details, see If you want to update attributes across multiple issues or merge requests, you can do it by bulk editing them, that is, editing them together. - + ## Bulk edit issues at the project level @@ -25,8 +25,12 @@ When bulk editing issues in a project, you can edit the following attributes: - Status (open/closed) - Assignee +- Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in + [GitLab Premium](https://about.gitlab.com/pricing/) 13.2.) **(PREMIUM)** - Milestone - Labels +- Health status ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218395) in + [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2.) **(ULTIMATE)** - Subscriptions To update multiple project issues at the same time: diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index d0cba729e35..65f1c59f4ca 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -13,6 +13,11 @@ GitLab offers integrated cluster creation for the following Kubernetes providers GitLab can also integrate with any standard Kubernetes provider, either on-premise or hosted. +NOTE: **Note:** +Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) +and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) +in a few clicks. + TIP: **Tip:** Every new Google Cloud Platform (GCP) account receives [$300 in credit upon sign up](https://console.cloud.google.com/freetrial), and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's @@ -23,7 +28,7 @@ Google Kubernetes Engine Integration. All you have to do is [follow this link](h Before [adding a Kubernetes cluster](#create-new-cluster) using GitLab, you need: - GitLab itself. Either: - - A GitLab.com [account](https://about.gitlab.com/pricing/#gitlab-com). + - A [GitLab.com account](https://about.gitlab.com/pricing/#gitlab-com). - A [self-managed installation](https://about.gitlab.com/pricing/#self-managed) with GitLab version 12.5 or later. This will ensure the GitLab UI can be used for cluster creation. - The following GitLab access: @@ -52,14 +57,10 @@ to manage the newly created cluster. NOTE: **Note:** Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. -When you install Helm into your cluster, the `tiller` service account -is created with `cluster-admin` privileges in the `gitlab-managed-apps` -namespace. - -This service account will be: - -- Added to the installed Helm Tiller. -- Used by Helm to install and run [GitLab managed applications](index.md#installing-applications). +The first time you install an application into your cluster, the `tiller` service +account is created with `cluster-admin` privileges in the +`gitlab-managed-apps` namespace. This service account will be used by Helm to +install and run [GitLab managed applications](index.md#installing-applications). Helm will also create additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application @@ -88,8 +89,8 @@ GitLab creates the following resources for RBAC clusters. | `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | | `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new cluster | | `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | | Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | | Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | | Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | @@ -103,8 +104,8 @@ GitLab creates the following resources for ABAC clusters. |:----------------------|:---------------------|:-------------------------------------|:---------------------------| | `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | | `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | | Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | | Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | | Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | @@ -126,7 +127,7 @@ arbitrary images as they effectively have root access. If you don't want to use GitLab Runner in privileged mode, either: - Use shared Runners on GitLab.com. They don't have this security issue. -- Set up your own Runners using configuration described at +- Set up your own Runners using the configuration described at [Shared Runners](../../gitlab_com/index.md#shared-runners). This involves: 1. Making sure that you don't have it installed via [the applications](index.md#installing-applications). @@ -135,23 +136,26 @@ If you don't want to use GitLab Runner in privileged mode, either: ## Create new cluster -New clusters can be created using GitLab for: +New clusters can be created using GitLab on Google Kubernetes Engine (GKE) or +Amazon Elastic Kubernetes Service (EKS) at the project, group, or instance level: -- [Google Kubernetes Engine (GKE)](add_gke_clusters.md). -- [Amazon Elastic Kubernetes Service (EKS)](add_eks_clusters.md). +1. Navigate to your: + - Project's **{cloud-gear}** **Operations > Kubernetes** page, for a project-level cluster. + - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. + - **{admin}** **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Click **Add Kubernetes cluster**. +1. Click the **Create new cluster** tab. +1. Click either **Amazon EKS** or **Google GKE**, and follow the instructions for your desired service: + - [Amazon EKS](add_eks_clusters.md#new-eks-cluster). + - [Google GKE](add_gke_clusters.md#creating-the-cluster-on-gke). ## Add existing cluster If you have an existing Kubernetes cluster, you can add it to a project, group, or instance. -For more information, see information for adding an: - -- [Existing Kubernetes cluster](#existing-kubernetes-cluster), including GKE clusters. -- [Existing EKS cluster](add_eks_clusters.md#existing-eks-cluster). - NOTE: **Note:** Kubernetes integration is not supported for arm64 clusters. See the issue -[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64044) for details. +[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838) for details. ### Existing Kubernetes cluster @@ -214,9 +218,9 @@ To add a Kubernetes cluster to your project, group, or instance: kind: ClusterRole name: cluster-admin subjects: - - kind: ServiceAccount - name: gitlab-admin - namespace: kube-system + - kind: ServiceAccount + name: gitlab-admin + namespace: kube-system ``` 1. Apply the service account and cluster role binding to your cluster: @@ -297,14 +301,15 @@ to install some [pre-defined applications](index.md#installing-applications). When connecting a cluster via GitLab integration, you may specify whether the cluster is RBAC-enabled or not. This will affect how GitLab interacts with the -cluster for certain operations. If you **did not** check the "RBAC-enabled cluster" +cluster for certain operations. If you did *not* check the **RBAC-enabled cluster** checkbox at creation time, GitLab will assume RBAC is disabled for your cluster when interacting with it. If so, you must disable RBAC on your cluster for the integration to work properly. - + -NOTE: **Note**: Disabling RBAC means that any application running in the cluster, +NOTE: **Note:** +Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](index.md#security-implications), and may not be desirable. @@ -320,17 +325,20 @@ kubectl create clusterrolebinding permissive-binding \ ## Enabling or disabling integration -After you have successfully added your cluster information, you can enable the -Kubernetes cluster integration: - -1. Click the **Enabled/Disabled** switch -1. Hit **Save** for the changes to take effect +The Kubernetes cluster integration enables after you have successfully either created +a new cluster or added an existing one. To disable Kubernetes cluster integration: -To disable the Kubernetes cluster integration, follow the same procedure. +1. Navigate to your: + - Project's **{cloud-gear}** **Operations > Kubernetes** page, for a project-level cluster. + - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. + - **{admin}** **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Click on the name of the cluster. +1. Click the **GitLab Integration** toggle. +1. Click **Save changes**. ## Removing integration -To remove the Kubernetes cluster integration from your project, either: +To remove the Kubernetes cluster integration from your project, first navigate to the **Advanced Settings** tab of the cluster details page and either: - Select **Remove integration**, to remove only the Kubernetes integration. - [From GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/26815), select diff --git a/doc/user/project/clusters/img/rbac.png b/doc/user/project/clusters/img/rbac.png Binary files differdeleted file mode 100644 index 517e4f7ca44..00000000000 --- a/doc/user/project/clusters/img/rbac.png +++ /dev/null diff --git a/doc/user/project/clusters/img/rbac_v13_1.png b/doc/user/project/clusters/img/rbac_v13_1.png Binary files differnew file mode 100644 index 00000000000..2772af9ff89 --- /dev/null +++ b/doc/user/project/clusters/img/rbac_v13_1.png diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 961a9fda5ff..ddcfd376d89 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -12,15 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in > GitLab 11.11 for [instances](../../instance/clusters/index.md). -GitLab provides many features with a Kubernetes integration. Kubernetes can be -integrated with projects, but also: - -- [Groups](../../group/clusters/index.md). -- [Instances](../../instance/clusters/index.md). - -NOTE: **Scalable app deployment with GitLab and Google Cloud Platform** -[Watch the webcast](https://about.gitlab.com/webcast/scalable-app-deploy/) and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) in a few clicks. - ## Overview Using the GitLab project Kubernetes integration, you can: @@ -28,17 +19,26 @@ Using the GitLab project Kubernetes integration, you can: - Use [Review Apps](../../../ci/review_apps/index.md). - Run [pipelines](../../../ci/pipelines/index.md). - [Deploy](#deploying-to-a-kubernetes-cluster) your applications. -- Detect and [monitor Kubernetes](#kubernetes-monitoring). +- Detect and [monitor Kubernetes](#monitoring-your-kubernetes-cluster). - Use it with [Auto DevOps](#auto-devops). - Use [Web terminals](#web-terminals). - Use [Deploy Boards](#deploy-boards-premium). **(PREMIUM)** - Use [Canary Deployments](#canary-deployments-premium). **(PREMIUM)** -- View [Logs](#logs). +- View [Logs](#viewing-pod-logs). - Run serverless workloads on [Kubernetes with Knative](serverless/index.md). +Besides integration at the project level, Kubernetes clusters can also be +integrated at the [group level](../../group/clusters/index.md) or +[GitLab instance level](../../instance/clusters/index.md). + +## Setting up + ### Supported cluster versions -GitLab is committed to support at least two production-ready Kubernetes minor versions at any given time. We regularly review the versions we support, and provide a four-month deprecation period before we remove support of a specific version. The range of supported versions is based on the evaluation of: +GitLab is committed to support at least two production-ready Kubernetes minor +versions at any given time. We regularly review the versions we support, and +provide a four-month deprecation period before we remove support of a specific +version. The range of supported versions is based on the evaluation of: - Our own needs. - The versions supported by major managed Kubernetes providers. @@ -55,80 +55,83 @@ Currently, GitLab supports the following Kubernetes versions: NOTE: **Note:** Some GitLab features may support versions outside the range provided here. -### Deploy Boards **(PREMIUM)** - -GitLab's Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes, -displaying the status of the pods in the deployment. Developers and other -teammates can view the progress and status of a rollout, pod by pod, in the -workflow they already use without any need to access Kubernetes. - -[Read more about Deploy Boards](../deploy_boards.md) - -### Canary Deployments **(PREMIUM)** - -Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) -and visualize your canary deployments right inside the Deploy Board, without -the need to leave GitLab. - -[Read more about Canary Deployments](../canary_deployments.md) +### Adding and removing clusters -### Logs - -GitLab makes it easy to view the logs of running pods in connected Kubernetes clusters. By displaying the logs directly in GitLab, developers can avoid having to manage console tools or jump to a different interface. - -[Read more about Kubernetes logs](kubernetes_pod_logs.md) +See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how +to: -### Kubernetes monitoring +- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service + (EKS) using GitLab's UI. +- Add an integration to an existing cluster from any Kubernetes platform. -Automatically detect and monitor Kubernetes metrics. Automatic monitoring of -[NGINX Ingress](../integrations/prometheus_library/nginx.md) is also supported. +### Multiple Kubernetes clusters -[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) +> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3 +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab core in 13.2. -### Auto DevOps +You can associate more than one Kubernetes cluster to your +project. That way you can have different clusters for different environments, +like dev, staging, production, and so on. -Auto DevOps automatically detects, builds, tests, deploys, and monitors your -applications. +Simply add another cluster, like you did the first time, and make sure to +[set an environment scope](#setting-the-environment-scope-premium) that will +differentiate the new cluster with the rest. -To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) -you will need the Kubernetes project integration enabled. +#### Setting the environment scope **(PREMIUM)** -[Read more about Auto DevOps](../../../topics/autodevops/index.md) +When adding more than one Kubernetes cluster to your project, you need to differentiate +them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the +[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. -NOTE: **Note** -Kubernetes clusters can be used without Auto DevOps. +The default environment scope is `*`, which means all jobs, regardless of their +environment, will use that cluster. Each scope can only be used by a single +cluster in a project, and a validation error will occur if otherwise. +Also, jobs that don't have an environment keyword set will not be able to access any cluster. -### Web terminals +For example, let's say the following Kubernetes clusters exist in a project: -> Introduced in GitLab 8.15. +| Cluster | Environment scope | +| ----------- | ----------------- | +| Development | `*` | +| Production | `production` | -When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) -support to your [environments](../../../ci/environments/index.md). This is based on the `exec` functionality found in -Docker and Kubernetes, so you get a new shell session within your existing -containers. To use this integration, you should deploy to Kubernetes using -the deployment variables above, ensuring any deployments, replica sets, and -pods are annotated with: +And the following environments are set in +[`.gitlab-ci.yml`](../../../ci/yaml/README.md): -- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` -- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` +```yaml +stages: + - test + - deploy -`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of -the CI variables. +test: + stage: test + script: sh test -You must be the project owner or have `maintainer` permissions to use terminals. Support is limited -to the first container in the first pod of your environment. +deploy to staging: + stage: deploy + script: make deploy + environment: + name: staging + url: https://staging.example.com/ -## Adding and removing clusters +deploy to production: + stage: deploy + script: make deploy + environment: + name: production + url: https://example.com/ +``` -See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how -to: +The result will then be: -- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service - (EKS) using GitLab's UI. -- Add an integration to an existing cluster from any Kubernetes platform. +- The Development cluster details will be available in the `deploy to staging` + job. +- The production cluster details will be available in the `deploy to production` + job. +- No cluster details will be available in the `test` job because it doesn't + define any environment. -## Cluster configuration +## Configuring your Kubernetes cluster After [adding a Kubernetes cluster](add_remove_clusters.md) to GitLab, read this section that covers important considerations for configuring Kubernetes clusters with GitLab. @@ -203,72 +206,6 @@ you can either: - Create an `A` record that points to the Ingress IP address with your domain provider. - Enter a wildcard DNS address using a service such as nip.io or xip.io. For example, `192.168.1.1.xip.io`. -### Setting the environment scope **(PREMIUM)** - -When adding more than one Kubernetes cluster to your project, you need to differentiate -them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the -[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. - -The default environment scope is `*`, which means all jobs, regardless of their -environment, will use that cluster. Each scope can only be used by a single -cluster in a project, and a validation error will occur if otherwise. -Also, jobs that don't have an environment keyword set will not be able to access any cluster. - -For example, let's say the following Kubernetes clusters exist in a project: - -| Cluster | Environment scope | -| ----------- | ----------------- | -| Development | `*` | -| Production | `production` | - -And the following environments are set in -[`.gitlab-ci.yml`](../../../ci/yaml/README.md): - -```yaml -stages: -- test -- deploy - -test: - stage: test - script: sh test - -deploy to staging: - stage: deploy - script: make deploy - environment: - name: staging - url: https://staging.example.com/ - -deploy to production: - stage: deploy - script: make deploy - environment: - name: production - url: https://example.com/ -``` - -The result will then be: - -- The Development cluster details will be available in the `deploy to staging` - job. -- The production cluster details will be available in the `deploy to production` - job. -- No cluster details will be available in the `test` job because it doesn't - define any environment. - -### Multiple Kubernetes clusters **(PREMIUM)** - -> Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. - -With GitLab Premium, you can associate more than one Kubernetes cluster to your -project. That way you can have different clusters for different environments, -like dev, staging, production, and so on. - -Simply add another cluster, like you did the first time, and make sure to -[set an environment scope](#setting-the-environment-scope-premium) that will -differentiate the new cluster with the rest. - ## Installing applications GitLab can install and manage some applications like Helm, GitLab Runner, Ingress, @@ -277,6 +214,19 @@ installing, upgrading, uninstalling, and troubleshooting applications for your project cluster, see [GitLab Managed Apps](../../clusters/applications.md). +## Auto DevOps + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. + +To make full use of Auto DevOps (Auto Deploy, Auto Review Apps, and +Auto Monitoring) you will need the Kubernetes project integration enabled. + +[Read more about Auto DevOps](../../../topics/autodevops/index.md) + +NOTE: **Note:** +Kubernetes clusters can be used without Auto DevOps. + ## Deploying to a Kubernetes cluster A Kubernetes cluster can be the destination for a deployment job. If @@ -325,10 +275,59 @@ For **non**-GitLab-managed clusters, the namespace can be customized using [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) in `.gitlab-ci.yml`. -NOTE: **Note:** When using a [GitLab-managed cluster](#gitlab-managed-clusters), the +NOTE: **Note:** +When using a [GitLab-managed cluster](#gitlab-managed-clusters), the namespaces are created automatically prior to deployment and [can not be customized](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +### Integrations + +#### Canary Deployments **(PREMIUM)** + +Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) +and visualize your canary deployments right inside the Deploy Board, without +the need to leave GitLab. + +[Read more about Canary Deployments](../canary_deployments.md) + +#### Deploy Boards **(PREMIUM)** + +GitLab's Deploy Boards offer a consolidated view of the current health and +status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes, +displaying the status of the pods in the deployment. Developers and other +teammates can view the progress and status of a rollout, pod by pod, in the +workflow they already use without any need to access Kubernetes. + +[Read more about Deploy Boards](../deploy_boards.md) + +#### Viewing pod logs + +GitLab makes it easy to view the logs of running pods in connected Kubernetes +clusters. By displaying the logs directly in GitLab, developers can avoid having +to manage console tools or jump to a different interface. + +[Read more about Kubernetes logs](kubernetes_pod_logs.md) + +#### Web terminals + +> Introduced in GitLab 8.15. + +When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) +support to your [environments](../../../ci/environments/index.md). This is based +on the `exec` functionality found in Docker and Kubernetes, so you get a new +shell session within your existing containers. To use this integration, you +should deploy to Kubernetes using the deployment variables above, ensuring any +deployments, replica sets, and pods are annotated with: + +- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` +- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` + +`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of +the CI variables. + +You must be the project owner or have `maintainer` permissions to use terminals. +Support is limited to the first container in the first pod of your environment. + ### Troubleshooting Before the deployment jobs starts, GitLab creates the following specifically for @@ -353,15 +352,23 @@ Reasons for failure include: [`environment:name`](../../../ci/environments/index.md#defining-environments). If your job has no `environment:name` set, it will not be passed the Kubernetes credentials. -NOTE: **NOTE:** +NOTE: **Note:** Project-level clusters upgraded from GitLab 12.0 or older may be configured in a way that causes this error. Ensure you deselect the [GitLab-managed cluster](#gitlab-managed-clusters) option if you want to manage namespaces and service accounts yourself. -## Monitoring your Kubernetes cluster **(ULTIMATE)** +## Monitoring your Kubernetes cluster + +Automatically detect and monitor Kubernetes metrics. Automatic monitoring of +[NGINX Ingress](../integrations/prometheus_library/nginx.md) is also supported. + +[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) + +### Visualizing cluster health -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/208224) to GitLab core in 13.2. When [Prometheus is deployed](#installing-applications), GitLab will automatically monitor the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start. diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 509be4ed0a8..ee642dc18cf 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -13,9 +13,9 @@ GitLab makes it easy to view the logs of running pods in [connected Kubernetes c By displaying the logs directly in GitLab in the **Log Explorer**, developers can avoid managing console tools or jumping to a different interface. -NOTE: **Kubernetes + GitLab** +NOTE: **Note:** +[Learn more about Kubernetes + GitLab](https://about.gitlab.com/solutions/kubernetes/). Everything you need to build, test, deploy, and run your application at scale. -[Learn more](https://about.gitlab.com/solutions/kubernetes/). ## Overview diff --git a/doc/user/project/clusters/runbooks/img/helm-install.png b/doc/user/project/clusters/runbooks/img/helm-install.png Binary files differdeleted file mode 100644 index e67cf317ec1..00000000000 --- a/doc/user/project/clusters/runbooks/img/helm-install.png +++ /dev/null diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 92ef35ad93f..a592d59f964 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -42,9 +42,6 @@ To create an executable runbook, you will need: - **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications. The simplest way to get started is to add a cluster using one of [GitLab's integrations](../add_remove_clusters.md#create-new-cluster). -- **Helm Tiller** - Helm is a package manager for Kubernetes and is required to - install all the other applications. It's installed in its own pod inside the - cluster which can run the Helm CLI in a safe environment. - **Ingress** - Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications. - **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user @@ -68,13 +65,8 @@ the components outlined above and the pre-loaded demo runbook. 1. Add a Kubernetes cluster to your project by following the steps outlined in [Create new cluster](../add_remove_clusters.md#create-new-cluster). -1. After the cluster has been provisioned in GKE, click the **Install** button - next to the **Helm Tiller** application to install Helm Tiller. -  - -1. After Helm Tiller has been installed successfully, click the **Install** button next - to the **Ingress** application. +1. Click the **Install** button next to the **Ingress** application to install Ingress.  diff --git a/doc/user/project/clusters/securing.md b/doc/user/project/clusters/securing.md new file mode 100644 index 00000000000..b4c20cb8dbc --- /dev/null +++ b/doc/user/project/clusters/securing.md @@ -0,0 +1,154 @@ +--- +stage: Defend +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Securing your deployed applications + +GitLab makes it easy to secure applications deployed in [connected Kubernetes clusters](index.md). +You can benefit from the protection of a [Web Application Firewall](../../../topics/web_application_firewall/quick_start_guide.md), +[Network Policies](../../../topics/autodevops/stages.md#network-policy), +or even [Container Host Security](../../clusters/applications.md#install-falco-using-gitlab-cicd). + +This page contains full end-to-end steps and instructions to connect your cluster to GitLab and +install these features, whether or not your applications are deployed through GitLab CI/CD. If you +use [Auto DevOps](../../../topics/autodevops/index.md) +to build and deploy your application with GitLab, see the documentation for the respective +[GitLab Managed Applications](../../clusters/applications.md) +above. + +## Overview + +At a high level, the required steps include the following: + +- Connect the cluster to GitLab. +- Set up one or more runners. +- Set up a cluster management project. +- Install a Web Application Firewall, Network Policies, and/or Container Host + Security. +- Install Prometheus to get statistics and metrics in the + [threat monitoring](../../application_security/threat_monitoring/) + dashboard. + +### Requirements + +Minimum requirements (depending on the GitLab Manage Application you want to install): + +- Your cluster is connected to GitLab (ModSecurity, Cilium, and Falco). +- At least one GitLab Runner is installed (Cilium and Falco only). + +### Understanding how GitLab Managed Apps are installed + +You install GitLab Managed Apps from the GitLab web interface with a one-click setup process. GitLab +uses Sidekiq (a background processing service) to facilitate this. + +```mermaid + sequenceDiagram + autonumber + GitLab->>+Sidekiq: Install a GitLab Managed App + Sidekiq->>+Kubernetes: Helm install + Kubernetes-->>-Sidekiq: Installation complete + Sidekiq-->>-GitLab: Refresh UI +``` + +NOTE: **Note:** +This diagram uses the term _Kubernetes_ for simplicity. In practice, Sidekiq connects to a Helm +Tiller daemon running in a pod in the cluster. + +Although this installation method is easier because it's a point-and-click action in the user +interface, it's inflexible and hard to debug. When something goes wrong, you can't see the +deployment logs. The Web Application Firewall feature uses this installation method. + +However, the next generation of GitLab Managed Apps V2 ([CI/CD-based GitLab Managed Apps](https://gitlab.com/groups/gitlab-org/-/epics/2103)) +don't use Sidekiq to deploy. All the applications are deployed using a GitLab CI/CD pipeline and +therefore GitLab Runners. + +```mermaid +sequenceDiagram + autonumber + GitLab->>+GitLab: Trigger pipeline + GitLab->>+Runner: Run deployment job + Runner->>+Kubernetes: Helm install + Kubernetes-->>-Runner: Installation is complete + Runner-->>-GitLab: Report job status and update pipeline +``` + +Debugging is easier because you have access to the raw logs of these jobs (the Helm Tiller output is +available as an artifact in case of failure) and the flexibility is much better. Since these +deployments are only triggered when a pipeline is running (most likely when there's a new commit in +the cluster management repository), every action has a paper trail and follows the classic merge +request workflow (approvals, merge, deploy). The Network Policy (Cilium) Managed App and Container +Host Security (Falco) are deployed with this model. + +## Connect the cluster to GitLab + +To deploy GitLab Managed Apps to your cluster, you must first +[add your cluster](add_remove_clusters.md) +to GitLab. Then [install](../../clusters/applications.md#installing-applications) +the Web Application Firewall from the project or group Kubernetes page. + +Note that your project doesn't have to be hosted or deployed through GitLab. You can manage a +cluster independent of the applications that use the cluster. + +## Set up a GitLab Runner + +To install CI/CD-based GitLab Managed Apps, a pipeline using a GitLab Runner must be running in +GitLab. You can [install a GitLab Runner](../../clusters/applications.md#gitlab-runner) +in the Kubernetes cluster added in the previous step, or use one of the shared runners provided by +GitLab if you're using GitLab.com. + +With your cluster connected to GitLab and a GitLab Runner in place, you can proceed to the next +steps and start installing the Cilium and Falco GitLab Managed Apps to secure your applications +hosted on this cluster. + +## Create a Cluster Management Project + +A [Cluster Management Project](../../clusters/management_project.md) +is a GitLab project that contains a `.gitlab-ci.yml` file to deploy GitLab Managed Apps to your +cluster. This project runs the required charts with the Kubernetes +[`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) +privileges. + +The creation of this project starts like any other GitLab project. Use an empty +project and add a `gitlab-ci.yml` file at the root, containing this template: + +```yaml +include: + - template: Managed-Cluster-Applications.gitlab-ci.yml +``` + +To make this project a Cluster Management Project, follow these +[instructions](../../clusters/management_project.md#selecting-a-cluster-management-project). +This project can be designated as such even if your application isn't hosted on GitLab. In this +case, create a new empty project where you can select your newly created Cluster Management Project. + +## Install GitLab Container Network Policy + +GitLab Container Network Policy is based on [Cilium](https://cilium.io/). To +install the Cilium GitLab Managed App, add a +`.gitlab/managed-apps/config.yaml` file to your Cluster Management project: + +```yaml +# possible values are gke, eks or you can leave it blank +clusterType: gke + +cilium: + installed: true +``` + +Your application doesn't have to be managed or deployed by GitLab to leverage this feature. +[Read more](../../clusters/applications.md#install-cilium-using-gitlab-cicd) +about configuring Container Network Policy. + +## Install GitLab Container Host Security + +Similarly, you can install Container Host Security, based on +[Falco](https://falco.org/), in your `.gitlab/managed-apps/config.yaml`: + +```yaml +falco: + installed: true +``` + +[Read more] about configuring Container Host Security. diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 15f7e14fda9..595d8fb3895 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -392,29 +392,19 @@ want to store your package: image: python:latest stages: - - deploy production: - stage: deploy - before_script: - - pip3 install awscli --upgrade - - pip3 install aws-sam-cli --upgrade - script: - - sam build - - sam package --output-template-file packaged.yaml --s3-bucket <S3_bucket_name> - - sam deploy --template-file packaged.yaml --stack-name gitlabpoc --s3-bucket <S3_bucket_name> --capabilities CAPABILITY_IAM --region us-east-1 - environment: production - ``` +``` Let’s examine the configuration file more closely: diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 45fb313d177..6af08b06294 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -51,8 +51,6 @@ To run Knative on GitLab, you will need: 1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. The simplest way to get started is to add a cluster using GitLab's [GKE integration](../add_remove_clusters.md). The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. -1. **Helm Tiller:** Helm is a package manager for Kubernetes and is required to install - Knative. 1. **GitLab Runner:** A runner is required to run the CI jobs that will deploy serverless applications or functions onto your cluster. You can install the GitLab Runner onto the existing Kubernetes cluster. See [Installing Applications](../index.md#installing-applications) for more information. @@ -80,8 +78,8 @@ To run Knative on GitLab, you will need: NOTE: **Note:** The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. **RBAC must be enabled.** -1. [Add a Kubernetes cluster](../add_remove_clusters.md) and [install Helm](../index.md#installing-applications). -1. Once Helm has been successfully installed, scroll down to the Knative app section. Enter the domain to be used with +1. [Add a Kubernetes cluster](../add_remove_clusters.md). +1. Select the **Applications** tab and scroll down to the Knative app section. Enter the domain to be used with your application/functions (e.g. `example.com`) and click **Install**.  @@ -143,24 +141,24 @@ You must do the following: labels: rbac.authorization.k8s.io/aggregate-to-edit: "true" rules: - - apiGroups: - - serving.knative.dev - resources: - - configurations - - configurationgenerations - - routes - - revisions - - revisionuids - - autoscalers - - services - verbs: - - get - - list - - create - - update - - delete - - patch - - watch + - apiGroups: + - serving.knative.dev + resources: + - configurations + - configurationgenerations + - routes + - revisions + - revisionuids + - autoscalers + - services + verbs: + - get + - list + - create + - update + - delete + - patch + - watch ``` Then run the following command: @@ -570,7 +568,7 @@ The simplest way to accomplish this is to use [Certbot to manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS. NOTE: **Note:** -The instructions below relate to installing and running Certbot on a Linux server and may not work on other operating systems. +The instructions below relate to installing and running Certbot on a Linux server that has Python 3 installed and may not work on other operating systems or with other versions of Python. 1. Install Certbot by running the [`certbot-auto` wrapper script](https://certbot.eff.org/docs/install.html#certbot-auto). @@ -580,7 +578,7 @@ The instructions below relate to installing and running Certbot on a Linux serve wget https://dl.eff.org/certbot-auto sudo mv certbot-auto /usr/local/bin/certbot-auto sudo chown root /usr/local/bin/certbot-auto - chmod 0755 /usr/local/bin/certbot-auto + sudo chmod 0755 /usr/local/bin/certbot-auto /usr/local/bin/certbot-auto --help ``` @@ -609,7 +607,7 @@ The instructions below relate to installing and running Certbot on a Linux serve using DNS challenge during authorization: ```shell - ./certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' + /usr/local/bin/certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' ``` Where `<namespace>` is the namespace created by GitLab for your serverless project (composed of `<project_name>-<project_id>-<environment>`) and @@ -835,7 +833,7 @@ The instructions below relate to installing and running Certbot on a Linux serve ## Using an older version of `gitlabktl` There may be situations where you want to run an older version of `gitlabktl`. This -requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml file.` +requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml` file. To set an older version, add `image:` to the `functions:deploy` block. For example: diff --git a/doc/user/project/code_intelligence.md b/doc/user/project/code_intelligence.md new file mode 100644 index 00000000000..e2c2cae3158 --- /dev/null +++ b/doc/user/project/code_intelligence.md @@ -0,0 +1,54 @@ +--- +type: reference +--- + +# Code Intelligence + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/1576) in GitLab 13.1. + +Code Intelligence adds code navigation features common to interactive +development environments (IDE), including: + +- Type signatures and symbol documentation. +- Go-to definition + +Code Intelligence is built into GitLab and powered by [LSIF](https://lsif.dev/) +(Language Server Index Format), a file format for precomputed code +intelligence data. + +## Configuration + +Enable code intelligence for a project by adding a GitLab CI/CD job to the project's +`.gitlab-ci.yml` which will generate the LSIF artifact: + +```yaml +code_navigation: + image: golang:1.14.0 + allow_failure: true # recommended + script: + - go get github.com/sourcegraph/lsif-go/cmd/lsif-go + - lsif-go + artifacts: + reports: + lsif: dump.lsif +``` + +The generated LSIF file must be less than 170MiB. + +After the job succeeds, code intelligence data can be viewed while browsing the code: + + + +## Language support + +Generating an LSIF file requires a language server indexer implementation for the +relevant language. + +| Language | Implementation | +|---|---| +| Go | [sourcegraph/lsif-go](https://github.com/sourcegraph/lsif-go) | +| JavaScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | +| TypeScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | + +View a complete list of [available LSIF indexers](https://lsif.dev/#implementations-server) on their website and +refer to their documentation to see how to generate an LSIF file for your specific language. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 6b81aea4b87..b35d04dfdfc 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -22,7 +22,7 @@ who is responsible for each file or path. ## Why is this useful? -Code Owners allows for a version controlled single source of +Code Owners allows for a version controlled, single source of truth file outlining the exact GitLab users or groups that own certain files or paths in a repository. Code Owners can be utilized in the merge request approval process which can streamline @@ -38,18 +38,18 @@ approval. You can use a `CODEOWNERS` file to specify users or [shared groups](members/share_project_with_groups.md) -that are responsible for certain files in a repository. +that are responsible for specific files and directories in a repository. -You can choose and add the `CODEOWNERS` file in three places: +You can choose to add the `CODEOWNERS` file in three places: - To the root directory of the repository - Inside the `.gitlab/` directory - Inside the `docs/` directory -The `CODEOWNERS` file is scoped to a branch, which means that with the -introduction of new files, the person adding the new content can +The `CODEOWNERS` file is scoped to a branch, which means that as +new files are introduced, the user adding the new content can specify themselves as a code owner, all before the new changes -get merged to the default branch. +get merged to the target branch. When a file matches multiple entries in the `CODEOWNERS` file, the users from last pattern matching the file are displayed on the @@ -67,13 +67,13 @@ The user that would show for `README.md` would be `@user2`. ## Approvals by Code Owners -Once you've set Code Owners to a project, you can configure it to +Once you've added Code Owners to a project, you can configure it to be used for merge request approvals: - As [merge request eligible approvers](merge_requests/merge_request_approvals.md#code-owners-as-eligible-approvers). - As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium). **(PREMIUM)** -NOTE: **Note**: +NOTE: **Note:** Developer or higher [permissions](../permissions.md) are required in order to approve a merge request. @@ -81,7 +81,7 @@ Once set, Code Owners are displayed in merge requests widgets:  -While the `CODEOWNERS` file can be used in addition to Merge Request [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules) +While the `CODEOWNERS` file can be used in addition to Merge Request [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules), it can also be used as the sole driver of merge request approvals (without using [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules)). To do so, create the file in one of the three locations specified above and @@ -89,28 +89,38 @@ set the code owners as required approvers for [protected branches](protected_bra Use [the syntax of Code Owners files](code_owners.md#the-syntax-of-code-owners-files) to specify the actual owners and granular permissions. -Using Code Owners in conjunction with [Protected Branches Approvals](protected_branches.md#protected-branches-approval-by-code-owners-premium) -will prevent any user who is not specified in the `CODEOWNERS` file from pushing changes -for the specified files/paths, even if their role is included in the **Allowed to push** column. -This allows for a more inclusive push strategy, as administrators don't have to restrict developers -from pushing directly to the protected branch, but can restrict pushing to certain -files where a review by Code Owners is required. +Using Code Owners in conjunction with [Protected Branches](protected_branches.md#protected-branches-approval-by-code-owners-premium) +will prevent any user who is not specified in the `CODEOWNERS` file from pushing +changes for the specified files/paths, even if their role is included in the +**Allowed to push** column. This allows for a more inclusive push strategy, as +administrators don't have to restrict developers from pushing directly to the +protected branch, but can restrict pushing to certain files where a review by +Code Owners is required. ## The syntax of Code Owners files Files can be specified using the same kind of patterns you would use -in the `.gitignore` file followed by the `@username` or email of one -or more users or by the `@name` of one or more groups that should -be owners of the file. Groups must be added as [members of the project](members/index.md), +in the `.gitignore` file followed by one or more of: + +- A user's `@username`. +- A user's email address. +- The `@name` of one or more groups that should be owners of the file. + +Groups must be added as [members of the project](members/index.md), or they will be ignored. -Starting in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/32432), you can now specify -groups or subgroups from the project's group hierarchy as potential code owners. +Starting in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/32432), +you can additionally specify groups or subgroups from the project's upper group +hierarchy as potential code owners, without having to invite them specifically +to the project. Groups outside the project's hierarchy or children beneath the +hierarchy must still be explicitly invited to the project in order to show as +Code Owners. -For example, consider the following hierarchy for a given project: +For example, consider the following hierarchy for the example project +`example_project`: ```plaintext -group >> sub-group >> sub-subgroup >> myproject >> file.md +group >> sub-group >> sub-subgroup >> example_project >> file.md ``` Any of the following groups would be eligible to be specified as code owners: @@ -119,7 +129,8 @@ Any of the following groups would be eligible to be specified as code owners: - `@group/sub-group` - `@group/sub-group/sub-subgroup` -In addition, any groups that have been invited to the project using the **Members** tool will also be recognized as eligible code owners. +In addition, any groups that have been invited to the project using the +**Members** tool will also be recognized as eligible code owners. The order in which the paths are defined is significant: the last pattern that matches a given path will be used to find the code @@ -129,7 +140,98 @@ Starting a line with a `#` indicates a comment. This needs to be escaped using `\#` to address files for which the name starts with a `#`. -Example `CODEOWNERS` file: +### Code Owners Sections **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. +> - It's deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It can be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-code-owner-sections-core-only). **(CORE ONLY)** + +Code Owner rules can be grouped into named sections. This allows for better +organization of broader categories of Code Owner rules to be applied. +Additionally, the usual guidance that only the last pattern matching the file is +applied is expanded such that the last pattern matching _for each section_ is +applied. + +For example, in a large organization, independent teams may have a common interest +in parts of the application, for instance, a payment processing company may have +"development", "security", and "compliance" teams looking after common parts of +the codebase. All three teams may need to approve changes. Although approval rules +make this possible, they apply to every merge request. Also, while Code Owners are +applied based on which files are changed, only one CODEOWNERS pattern can match per +file path. + +Using `CODEOWNERS` sections allows multiple teams that only need to approve certain +changes, to set their own independent patterns by specifying discrete sections in the +`CODEOWNERS` file. The section rules may be used for shared paths so that multiple +teams can be added as reviewers. + +Sections can be added to `CODEOWNERS` files as a new line with the name of the +section inside square brackets. Every entry following it is assigned to that +section. The following example would create 2 Code Owner rules for the "README +Owners" section: + +```plaintext +[README Owners] +README.md @user1 @user 2 +internal/README.md @user2 +``` + +Multiple sections can be used, even with matching file or directory patterns. +Reusing the same section name will group the results together under the same +section, with the most specific rule or last matching pattern being used. For +example, consider the following entries in a `CODEOWNERS` file: + +```plaintext +[Documentation] +ee/docs @gl-docs +docs @gl-docs + +[Database] +README.md @gl-database +model/db @gl-database + +[DOCUMENTATION] +README.md @gl-docs +``` + +This will result in 3 entries under the "Documentation" section header, and 2 +entries under "Database". Case is not considered when combining sections, so in +this example, entries defined under the sections "Documentation" and +"DOCUMENTATION" would be combined into one, using the case of the first instance +of the section encountered in the file. + +When assigned to a section, each code owner rule displayed in merge requests +widget is sorted under a "section" label. In the screenshot below, we can see +the rules for "Groups" and "Documentation" sections: + + + +#### Enable or disable Code Owner Sections **(CORE ONLY)** + +Sections is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it for your instance. + +To disable it: + +```ruby +Feature.disable(:sectional_codeowners) +``` + +To enable it: + +```ruby +Feature.enable(:sectional_codeowners) +``` + +CAUTION: **Caution:** +Disabling Sections will **not** refresh Code Owner Approval Rules on existing merge requests. + +## Example `CODEOWNERS` file ```plaintext # This is an example of a code owners file @@ -185,4 +287,17 @@ lib/ @lib-owner # If the path contains spaces, these need to be escaped like this: path\ with\ spaces/ @space-owner + +# Code Owners section: +[Documentation] +ee/docs @gl-docs +docs @gl-docs + +[Database] +README.md @gl-database +model/db @gl-database + +# This section will be joined with the [Documentation] section previously defined: +[DOCUMENTATION] +README.md @gl-docs ``` diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 10f0281d0eb..b7daca567f4 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -11,7 +11,7 @@ type: howto > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199370) from **Settings > Repository** in GitLab 12.9. > - [Added `write_registry` scope](https://gitlab.com/gitlab-org/gitlab/-/issues/22743) in GitLab 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29280) from **Settings > CI / CD** in GitLab 12.10.1. -> - [Added package registry scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) from **Settings > CI / CD** in GitLab 13.0. +> - [Added package registry scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. Deploy tokens allow you to download (`git clone`) or push and pull packages and container registry images of a project without having a user and a password. @@ -46,15 +46,17 @@ respective **Revoke** button under the 'Active deploy tokens' area. ## Limiting scopes of a deploy token -Deploy tokens can be created with two different scopes that allow various +Deploy tokens can be created with different scopes that allow various actions that a given token can perform. The available scopes are depicted in -the following table. +the following table along with GitLab version it was introduced in. -| Scope | Description | -| ----- | ----------- | -| `read_repository` | Allows read-access to the repository through `git clone` | -| `read_registry` | Allows read-access to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | -| `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | +| Scope | Description | Introduced in GitLab Version | +| ----- | ----------- | ------ | +| `read_repository` | Allows read-access to the repository through `git clone` | 10.7 | +| `read_registry` | Allows read-access to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | 10.7 | +| `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | 12.10 | +| `read_package_registry` | Allows read access to the package registry. | 13.0 | +| `write_package_registry` | Allows write access to the package registry. | 13.0 | ## Deploy token custom username @@ -96,6 +98,8 @@ pull images from your Container Registry. ### Push Container Registry images +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22743) in GitLab 12.10. + To push the container registry images, you'll need to: 1. Create a Deploy Token with `write_registry` as a scope. @@ -111,6 +115,8 @@ push images to your Container Registry. ### Read or pull packages +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. + To pull packages in the GitLab package registry, you'll need to: 1. Create a Deploy Token with `read_package_registry` as a scope. @@ -119,6 +125,8 @@ To pull packages in the GitLab package registry, you'll need to: ### Push or upload packages +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. + To upload packages in the GitLab package registry, you'll need to: 1. Create a Deploy Token with `write_package_registry` as a scope. @@ -151,8 +159,7 @@ apply consistently when cloning the repository of related projects. There's a special case when it comes to Deploy Tokens. If a user creates one named `gitlab-deploy-token`, the username and token of the Deploy Token will be automatically exposed to the CI/CD jobs as environment variables: `CI_DEPLOY_USER` and -`CI_DEPLOY_PASSWORD`, respectively. With the GitLab Deploy Token, the -`read_registry` and `write_registry` scopes are implied. +`CI_DEPLOY_PASSWORD`, respectively. After you create the token, you can login to the Container Registry using those variables: diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 0f90c321a14..aa5987bf5f9 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -81,7 +81,7 @@ changes you made after picking the template and return it to its initial status.  -## Setting a default template for merge requests and issues **(STARTER)** +## Setting a default template for merge requests and issues **(STARTER)** > - This feature was introduced before [description templates](#overview) and is available in [GitLab Starter](https://about.gitlab.com/pricing/). It can be enabled in the project's settings. > - Templates for issues were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28) in GitLab EE 8.1. diff --git a/doc/user/project/highlighting.md b/doc/user/project/highlighting.md index 2bdb0ae2706..a2740294e62 100644 --- a/doc/user/project/highlighting.md +++ b/doc/user/project/highlighting.md @@ -1,6 +1,11 @@ # Syntax Highlighting -GitLab provides syntax highlighting on all files and snippets through the [Rouge](https://rubygems.org/gems/rouge) rubygem. It will try to guess what language to use based on the file extension, which most of the time is sufficient. +GitLab provides syntax highlighting on all files through the [Rouge](https://rubygems.org/gems/rouge) Ruby gem. It will try to guess what language to use based on the file extension, which most of the time is sufficient. + +NOTE: **Note:** +The [Web IDE](web_ide/index.md) and [Snippets](../snippets.md) use [Monaco Editor](https://microsoft.github.io/monaco-editor/) +for text editing, which internally uses the [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) +library for syntax highlighting. If GitLab is guessing wrong, you can override its choice of language using the `gitlab-language` attribute in `.gitattributes`. For example, if you are working in a Prolog project and using the `.pl` file extension (which would normally be highlighted as Perl), you can add the following to your `.gitattributes` file: @@ -27,3 +32,6 @@ To disable highlighting entirely, use `gitlab-language=text`. Lots more fun shen ``` Please note that these configurations will only take effect when the `.gitattributes` file is in your default branch (usually `master`). + +NOTE: **Note:** +The Web IDE does not support `.gitattribute` files, but it's [planned for a future release](https://gitlab.com/gitlab-org/gitlab/-/issues/22014). diff --git a/doc/user/project/img/bulk-editing.png b/doc/user/project/img/bulk-editing.png Binary files differdeleted file mode 100644 index 8ae649e5020..00000000000 --- a/doc/user/project/img/bulk-editing.png +++ /dev/null diff --git a/doc/user/project/img/bulk-editing_v13_2.png b/doc/user/project/img/bulk-editing_v13_2.png Binary files differnew file mode 100644 index 00000000000..871cb02c01f --- /dev/null +++ b/doc/user/project/img/bulk-editing_v13_2.png diff --git a/doc/user/project/img/code_intelligence_v13_1.png b/doc/user/project/img/code_intelligence_v13_1.png Binary files differnew file mode 100644 index 00000000000..0dff27bab43 --- /dev/null +++ b/doc/user/project/img/code_intelligence_v13_1.png diff --git a/doc/user/project/img/sectional_code_owners_v13.2.png b/doc/user/project/img/sectional_code_owners_v13.2.png Binary files differnew file mode 100644 index 00000000000..894771c26af --- /dev/null +++ b/doc/user/project/img/sectional_code_owners_v13.2.png diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index 0d6e059f1cf..3838289aec4 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -105,7 +105,7 @@ back to both GitLab and GitHub when completed. 1. The result of the job will be visible directly from the pipeline view: -  +  NOTE: **Note:** If you don't commit very often to your project, you may want to use diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png Binary files differindex 4ab42485d0b..3c1dc44df93 100644 --- a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png +++ b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png Binary files differindex 6278cb5f970..d98ad2aaa6e 100644 --- a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png +++ b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png b/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png Binary files differnew file mode 100644 index 00000000000..9cbffe2bb36 --- /dev/null +++ b/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png Binary files differdeleted file mode 100644 index bf9728e0311..00000000000 --- a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png +++ /dev/null diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index 0b8807bb9b3..395cca4726d 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -40,6 +40,8 @@ Make sure you have the integration set up before trying to import Jira issues. ## Import Jira issues to GitLab +> New import form [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216145) in GitLab 13.2. + To import Jira issues to a GitLab project, follow the steps below. NOTE: **Note:** @@ -47,27 +49,34 @@ Importing Jira issues is done as an asynchronous background job, which may result in delays based on import queues load, system load, or other factors. Importing large projects may take several minutes depending on the size of the import. -1. On the **{issues}** **Issues** page, click the **Import Issues** (**{import}**) button. -1. Select **Import from Jira**. - This option is only visible if you have the [correct permissions](#permissions). +1. On the **{issues}** **Issues** page, click **Import Issues** (**{import}**) **> Import from Jira**.  + The **Import from Jira** option is only visible if you have the [correct permissions](#permissions). + The following form appears. + If you've previously set up the [Jira integration](../integrations/jira.md), you can now see + the Jira projects that you have access to in the dropdown. + +  -  +1. Click the **Import from** dropdown and select the Jira project that you wish to import issues from. - If you've previously set up the [Jira integration](../integrations/jira.md), you now see the Jira - projects that you have access to in the dropdown. + In the **Jira-GitLab user mapping template** section, the table shows to which GitLab users your Jira + users will be mapped. + If it wasn't possible to match a Jira user with a GitLab user, the dropdown defaults to the user + conducting the import. -1. Select the Jira project that you wish to import issues from. +1. To change any of the suggested mappings, click the dropdown in the **GitLab username** column and + select the user you want to map to each Jira user. -  + The dropdown may not show all the users, so use the search bar to find a specific + user in this GitLab project. + +1. Click **Continue**. You're presented with a confirmation that import has started. -1. Click **Import Issues**. You're presented with a confirmation that import has started. While the import is running in the background, you can navigate away from the import status page to the issues page, and you'll see the new issues appearing in the issues list. -1. To check the status of your import, go back to the Jira import page. - -  +1. To check the status of your import, go to the Jira import page again. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 3a4e240fb6c..1e71674c16f 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -78,7 +78,7 @@ When you create a project in GitLab, you'll have access to a large number of timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more - [Kubernetes cluster integration](clusters/index.md): Connecting your GitLab project with a Kubernetes cluster - - [Feature Flags](operations/feature_flags.md): Feature flags allow you to ship a project in + - [Feature Flags](../../operations/feature_flags.md): Feature flags allow you to ship a project in different flavors by dynamically toggling certain functionality **(PREMIUM)** - [GitLab Pages](pages/index.md): Build, test, and deploy your static website with GitLab Pages @@ -104,6 +104,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Dependency List](../application_security/dependency_list/index.md): view project dependencies. **(ULTIMATE)** - [Requirements](requirements/index.md): Requirements allow you to create criteria to check your products against. **(ULTIMATE)** - [Static Site Editor](static_site_editor/index.md): quickly edit content on static websites without prior knowledge of the codebase or Git commands. +- [Code Intelligence](code_intelligence.md): code navigation features. ### Project integrations @@ -172,6 +173,24 @@ Read through the documentation on [project settings](settings/index.md). - [Export a project from GitLab](settings/import_export.md#exporting-a-project-and-its-data) - [Importing and exporting projects between GitLab instances](settings/import_export.md) +## Remove a project + +To remove a project, first navigate to the home page for that project. + +1. Navigate to **Settings > General**. +1. Expand the **Advanced** section. +1. Scroll down to the **Remove project** section. +1. Click **Remove project** +1. Confirm this action by typing in the expected text. + +### Delayed removal **(PREMIUM)** + +By default, clicking to remove a project is followed by a seven day delay. Admins can restore the project during this period of time. +This delay [may be changed by an admin](../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). + +Admins can view all projects pending deletion. If you're an administrator, go to the top navigation bar, click **Projects > Your projects**, and then select the **Removed projects** tab. +From this tab an admin can restore any project. + ## CI/CD for external repositories **(PREMIUM)** Instead of importing a repository directly to GitLab, you can connect your repository @@ -242,14 +261,52 @@ When [renaming a user](../profile/index.md#changing-your-username), ## Use your project as a Go package -Any project can be used as a Go package including private projects in subgroups. -GitLab responds correctly to `go get` and `godoc.org` discovery requests, -including the [`go-import`](https://golang.org/cmd/go/#hdr-Remote_import_paths) -and [`go-source`](https://github.com/golang/gddo/wiki/Source-Code-Links) meta -tags, respectively. To use packages hosted in private projects with the `go get` -command, use a [`.netrc` file](https://ec.haxx.se/usingcurl-netrc.html) and a -[personal access token](../profile/personal_access_tokens.md) in the password -field. +Any project can be used as a Go package. GitLab responds correctly to `go get` +and `godoc.org` discovery requests, including the +[`go-import`](https://golang.org/cmd/go/#hdr-Remote_import_paths) and +[`go-source`](https://github.com/golang/gddo/wiki/Source-Code-Links) meta tags. + +Private projects, including projects in subgroups, can be used as a Go package, +but may require configuration to work correctly. GitLab will respond correctly +to `go get` discovery requests for projects that *are not* in subgroups, +regardless of authentication or authorization. +[Authentication](#authenticate-go-requests) is required to use a private project +in a subgroup as a Go package. Otherwise, GitLab will truncate the path for +private projects in subgroups to the first two segments, causing `go get` to +fail. + +GitLab implements its own Go proxy. This feature must be enabled by an +administrator and requires additional configuration. See [GitLab Go +Proxy](../packages/go_proxy/index.md). + +### Disable Go module features for private projects + +In Go 1.12 and later, Go queries module proxies and checksum databases in the +process of [fetching a +module](../../development/go_guide/dependencies.md#fetching). This can be +selectively disabled with `GOPRIVATE` (disable both), +[`GONOPROXY`](../../development/go_guide/dependencies.md#proxies) (disable proxy +queries), and [`GONOSUMDB`](../../development/go_guide/dependencies.md#fetching) +(disable checksum queries). + +`GOPRIVATE`, `GONOPROXY`, and `GONOSUMDB` are comma-separated lists of Go +modules and Go module prefixes. For example, +`GOPRIVATE=gitlab.example.com/my/private/project` will disable queries for that +one project, but `GOPRIVATE=gitlab.example.com` will disable queries for *all* +projects on GitLab.com. Go will not query module proxies if the module name or a +prefix of it appears in `GOPRIVATE` or `GONOPROXY`. Go will not query checksum +databases if the module name or a prefix of it appears in `GONOPRIVATE` or +`GONOSUMDB`. + +### Authenticate Go requests + +To authenticate requests to private projects made by Go, use a [`.netrc` +file](https://ec.haxx.se/usingcurl-netrc.html) and a [personal access +token](../profile/personal_access_tokens.md) in the password field. **This only +works if your GitLab instance can be accessed with HTTPS.** The `go` command +will not transmit credentials over insecure connections. This will authenticate +all HTTPS requests made directly by Go but will not authenticate requests made +through Git. For example: @@ -259,6 +316,24 @@ login <gitlab_user_name> password <personal_access_token> ``` +NOTE: **Note:** +On Windows, Go reads `~/_netrc` instead of `~/.netrc`. + +### Authenticate Git fetches + +If a module cannot be fetched from a proxy, Go will fall back to using Git (for +GitLab projects). Git will use `.netrc` to authenticate requests. Alternatively, +Git can be configured to embed specific credentials in the request URL, or to +use SSH instead of HTTPS (as Go always uses HTTPS to fetch Git repositories): + +```shell +# embed credentials in any request to GitLab.com: +git config --global url."https://${user}:${personal_access_token}@gitlab.example.com".insteadOf "https://gitlab.example.com" + +# use SSH instead of HTTPS: +git config --global url."git@gitlab.example.com".insteadOf "https://gitlab.example.com" +``` + ## Access project page with project ID > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53671) in GitLab 11.8. diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 3cc76beb323..4d646ee2f79 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -271,6 +271,27 @@ you defined. | `week` | 4 | | `month` | 12 | +#### `query.period_field` + +Define the timestamp field used to group "issuables". + +Supported values are: + +- `created_at` (default): Group data using the `created_at` field. +- `closed_at`: Group data using the `closed_at` field (for issues only). +- `merged_at`: Group data using the `merged_at` field (for merge requests only). + +The `period_field` is automatically set to: + +- `closed_at` if `query.issuable_state` is `closed` +- `merged_at` if `query.issuable_state` is `merged` +- `created_at` otherwise + +NOTE: **Note:** +Until [this bug](https://gitlab.com/gitlab-org/gitlab/-/issues/26911), is resolved, you may see `created_at` +in place of `merged_at`. +`created_at` will be used instead. + ### `projects` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10904) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.4. diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index de43c504b05..6d44c56743e 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -6,7 +6,6 @@ in the table below. | Field | Description | | ----- | ----------- | -| `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Bugzilla which is being linked to this GitLab project. Note that the `project_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | | `issues_url` | The URL to the issue in Bugzilla project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | | `new_issue_url` | This is the URL to create a new issue in Bugzilla for the project linked to this GitLab project. Note that the `new_issue_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index 4eaf3a0d4b4..7d15ae82b6f 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -11,8 +11,6 @@ To enable the Custom Issue Tracker integration in a project: | Field | Description | | --------------- | ----------- | - | **Title** | A title for the issue tracker (for example, to differentiate between instances). | - | **Description** | A name for the issue tracker (for example, to differentiate between instances). | | **Project URL** | The URL to the project in the custom issue tracker. | | **Issues URL** | The URL to the issue in the issue tracker project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. For example, `https://customissuetracker.com/project-name/:id`. | | **New issue URL** | Currently unused. Will be changed in a future release. | diff --git a/doc/user/project/integrations/generic_alerts.md b/doc/user/project/integrations/generic_alerts.md index 57c1e54e48c..3490a3f2b9e 100644 --- a/doc/user/project/integrations/generic_alerts.md +++ b/doc/user/project/integrations/generic_alerts.md @@ -18,16 +18,21 @@ create an issue with the payload in the body of the issue. You can always The entire payload will be posted in the issue discussion as a comment authored by the GitLab Alert Bot. -NOTE: **Note** -In GitLab versions 13.1 and greater, you can configure [External Prometheus instances](prometheus.md#external-prometheus-instances) to use this endpoint. +NOTE: **Note:** +In GitLab versions 13.1 and greater, you can configure +[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances) +to use this endpoint. ## Setting up generic alerts -To set up the generic alerts integration: +To obtain credentials for setting up a generic alerts integration: -1. Navigate to **Settings > Integrations** in a project. -1. Click on **Alerts endpoint**. -1. Toggle the **Active** alert setting. The `URL` and `Authorization Key` for the webhook configuration can be found there. +- Sign in to GitLab as a user with maintainer [permissions](../../permissions.md) for a project. +- Navigate to the **Operations** page for your project, depending on your installed version of GitLab: + - *In GitLab versions 13.1 and greater,* navigate to **{settings}** **Settings > Operations** in your project. + - *In GitLab versions prior to 13.1,* navigate to **{settings}** **Settings > Integrations** in your project. GitLab will display a banner encouraging you to enable the Alerts endpoint in **{settings}** **Settings > Operations** instead. +- Click **Alerts endpoint**. +- Toggle the **Active** alert setting to display the **URL** and **Authorization Key** for the webhook configuration. ## Customizing the payload @@ -44,6 +49,14 @@ You can customize the payload by sending the following parameters. All fields ot | `severity` | String | The severity of the alert. Must be one of `critical`, `high`, `medium`, `low`, `info`, `unknown`. Default is `critical`. | | `fingerprint` | String or Array | The unique identifier of the alert. This can be used to group occurrences of the same alert. | +You can also add custom fields to the alert's payload. The values of extra parameters +are not limited to primitive types, such as strings or numbers, but can be a nested +JSON object. For example: + +```json +{ "foo": { "bar": { "baz": 42 } } } +``` + TIP: **Payload size:** Ensure your requests are smaller than the [payload application limits](../../../administration/instance_limits.md#generic-alert-json-payloads). @@ -70,6 +83,42 @@ Example payload: "monitoring_tool": "value", "hosts": "value", "severity": "high", - "fingerprint": "d19381d4e8ebca87b55cda6e8eee7385" + "fingerprint": "d19381d4e8ebca87b55cda6e8eee7385", + "foo": { + "bar": { + "baz": 42 + } + } } ``` + +## Triggering test alerts + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab Core in 13.2. + +After a [project maintainer or owner](#setting-up-generic-alerts) +[configures generic alerts](#setting-up-generic-alerts), you can trigger a +test alert to confirm your integration works properly. + +1. Sign in as a user with Developer or greater [permissions](../../../user/permissions.md). +1. Navigate to **{settings}** **Settings > Operations** in your project. +1. Click **Alerts endpoint** to expand the section. +1. Enter a sample payload in **Alert test payload** (valid JSON is required). +1. Click **Test alert payload**. + +GitLab displays an error or success message, depending on the outcome of your test. + +## Automatic grouping of identical alerts **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214557) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +In GitLab versions 13.2 and greater, GitLab groups alerts based on their payload. +When an incoming alert contains the same payload as another alert (excluding the +`start_time` and `hosts` attributes), GitLab groups these alerts together and +displays a counter on the +[Alert Management List](../operations/alert_management.md#alert-management-list) +and details pages. + +If the existing alert is already `resolved`, then a new alert will be created instead. + + diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index f0977a4ea76..416996fb629 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -14,7 +14,7 @@ and is automatically configured on [GitHub import](../../../integration/github.m ### Complete these steps on GitHub -This integration requires a [GitHub API token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) +This integration requires a [GitHub API token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with `repo:status` access granted: 1. Go to your "Personal access tokens" page at <https://github.com/settings/tokens> diff --git a/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png b/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png Binary files differnew file mode 100644 index 00000000000..5d530a80421 --- /dev/null +++ b/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png diff --git a/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png b/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png Binary files differnew file mode 100644 index 00000000000..c3a391b06c7 --- /dev/null +++ b/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png diff --git a/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png b/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png Binary files differnew file mode 100644 index 00000000000..0cf58433b25 --- /dev/null +++ b/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png diff --git a/doc/user/project/integrations/img/jira_service_page_v12_2.png b/doc/user/project/integrations/img/jira_service_page_v12_2.png Binary files differdeleted file mode 100644 index ba7dad9b438..00000000000 --- a/doc/user/project/integrations/img/jira_service_page_v12_2.png +++ /dev/null diff --git a/doc/user/project/integrations/img/metrics_settings_button_v13_2.png b/doc/user/project/integrations/img/metrics_settings_button_v13_2.png Binary files differnew file mode 100644 index 00000000000..d649f77eded --- /dev/null +++ b/doc/user/project/integrations/img/metrics_settings_button_v13_2.png diff --git a/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png b/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png Binary files differnew file mode 100644 index 00000000000..b6ec08f492d --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png diff --git a/doc/user/project/integrations/img/prometheus_service_configuration.png b/doc/user/project/integrations/img/prometheus_service_configuration.png Binary files differdeleted file mode 100644 index a38d1bce197..00000000000 --- a/doc/user/project/integrations/img/prometheus_service_configuration.png +++ /dev/null diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 442f3229de2..541c65041ad 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -55,29 +55,41 @@ In order to enable the Jira service in GitLab, you need to first configure the p > **Notes:** > -> - The currently supported Jira versions are `v6.x, v7.x, v8.x` . GitLab 7.8 or -> higher is required. -> - GitLab 8.14 introduced a new way to integrate with Jira which greatly simplified -> the configuration options you have to enter. If you are using an older version, -> [follow this documentation](https://gitlab.com/gitlab-org/gitlab/blob/8-13-stable-ee/doc/project_services/jira.md). +> - The supported Jira versions are `v6.x`, `v7.x`, and `v8.x`. > - In order to support Oracle's Access Manager, GitLab will send additional cookies > to enable Basic Auth. The cookie being added to each request is `OBBasicAuth` with > a value of `fromDialog`. To enable the Jira integration in a project, navigate to the -[Integrations page](overview.md#accessing-integrations), click -the **Jira** service, and fill in the required details on the page as described -in the table below. +[Integrations page](overview.md#accessing-integrations) and click +the **Jira** service. + +Select **Enable integration**. + +Select a **Trigger** action. This determines whether a mention of a Jira issue in GitLab commits, merge requests, or both, should link the Jira issue back to that source commit/MR and transition the Jira issue, if indicated. + +To include a comment on the Jira issue when the above referene is made in GitLab, check **Enable comments**. + +Enter the further details on the page as described in the following table. | Field | Description | | ----- | ----------- | | `Web URL` | The base URL to the Jira instance web interface which is being linked to this GitLab project. E.g., `https://jira.example.com`. | | `Jira API URL` | The base URL to the Jira instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. Leave this field blank (or use the same value of `Web URL`) if using **Jira Cloud**. | -| `Username/Email` | Created when [configuring Jira step](#configuring-jira). Use `username` for **Jira Server** or `email` for **Jira Cloud**. | -| `Password/API token` |Created in [configuring Jira step](#configuring-jira). Use `password` for **Jira Server** or `API token` for **Jira Cloud**. | -| `Transition ID` | This is the ID of a transition that moves issues to the desired state. It is possible to insert transition ids separated by `,` or `;` which means the issue will be moved to each state after another using the given order. **Closing Jira issues via commits or Merge Requests won't work if you don't set the ID correctly.** | +| `Username or Email` | Created in [configuring Jira](#configuring-jira) step. Use `username` for **Jira Server** or `email` for **Jira Cloud**. | +| `Password/API token` |Created in [configuring Jira](#configuring-jira) step. Use `password` for **Jira Server** or `API token` for **Jira Cloud**. | +| `Transition ID` | Required for closing Jira issues via commits or merge requests. This is the ID of a transition in Jira that moves issues to a desired state. (See [Obtaining a transition ID](#obtaining-a-transition-id).) If you insert multiple transition IDs separated by `,` or `;`, the issue is moved to each state, one after another, using the given order. | + +To enable users to view Jira issues inside GitLab, select **Enable Jira issues** and enter a project key. **(PREMIUM)** + +CAUTION: **Caution:** +If you enable Jira issues with the setting above, all users that have access to this GitLab project will be able to view all issues from the specified Jira project. + +When you have configured all settings, click **Test settings and save changes**. -### Obtaining a transition ID +Your GitLab project can now interact with all Jira projects in your instance and the project now displays a Jira link that opens the Jira project. + +#### Obtaining a transition ID In the most recent Jira user interface, you can no longer see transition IDs in the workflow administration UI. You can get the ID you need in either of the following ways: @@ -90,19 +102,11 @@ administration UI. You can get the ID you need in either of the following ways: Note that the transition ID may vary between workflows (e.g., bug vs. story), even if the status you are changing to is the same. -After saving the configuration, your GitLab project will be able to interact -with all Jira projects in your Jira instance and you'll see the Jira link on the GitLab project pages that takes you to the appropriate Jira project. - - - -### Disabling comments on Jira issues - -When you reference a Jira issue, it will always link back to the source commit/MR in GitLab, however, you can control whether GitLab will also cross-post a comment to the Jira issue. That functionality is enabled by default. +#### Disabling comments on Jira issues -To disable the automated commenting on Jira issues: +You can continue to have GitLab cross-link a source commit/MR with a Jira issue while disabling the comment added to the issue. -1. Open the [Integrations page](overview.md#accessing-integrations) and select **Jira**. -1. In the **Event Action** section, uncheck **Comment**. +See the [Configuring GitLab](#configuring-gitlab) section and uncheck the **Enable comments** setting. ## Jira issues @@ -111,7 +115,7 @@ By now you should have [configured Jira](#configuring-jira) and enabled the you should be able to reference and close Jira issues by just mentioning their ID in GitLab commits and merge requests. -### Referencing Jira Issues +### Reference Jira issues When GitLab project has Jira issue tracker configured and enabled, mentioning Jira issue in GitLab will automatically add a comment in Jira issue with the @@ -138,7 +142,7 @@ For example, the following commit will reference the Jira issue with `PROJECT-1` git commit -m "PROJECT-1 Fix spelling and grammar" ``` -### Closing Jira Issues +### Close Jira issues Jira issues can be closed directly from GitLab by using trigger words in commits and merge requests. When a commit which contains the trigger word @@ -162,8 +166,6 @@ where `PROJECT-1` is the ID of the Jira issue. > [project settings](img/jira_project_settings.png). > - The Jira issue will not be transitioned if it has a resolution. -### Jira issue closing example - Let's consider the following example: 1. For the project named `PROJECT` in Jira, we implemented a new feature @@ -185,6 +187,45 @@ with a link to the commit that resolved the issue.  +### View Jira issues **(PREMIUM)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3622) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +You can browse and search issues from a selected Jira project directly in GitLab. This requires [configuration](#configuring-gitlab) in GitLab by an administrator. + + + +From the **Jira Issues** menu, click **Issues List**. The issue list defaults to sort by **Created date**, with the newest issues listed at the top. You can change this to **Last updated**. + +Issues are grouped into tabs based on their [Jira status](https://confluence.atlassian.com/adminjiraserver070/defining-status-field-values-749382903.html). + +- The **Open** tab displays all issues with a Jira status in any category other than Done. +- The **Closed** tab displays all issues with a Jira status categorized as Done. +- The **All** tab displays all issues of any status. + +Click an issue title to open its original Jira issue page for full details. + +#### Search and filter the issues list + +To refine the list of issues, use the search bar to search for any text +contained in an issue summary (title) or description. + +You can also filter by labels, status, reporter, and assignee using URL parameters. +Enhancements to be able to use these through the user interface are [planned](https://gitlab.com/groups/gitlab-org/-/epics/3622). + +- To filter issues by `labels`, specify one or more labels as part of the `labels[]` +parameter in the URL. When using multiple labels, only issues that contain all specified +labels are listed. `/-/integrations/jira/issues?labels[]=backend&labels[]=feature&labels[]=QA` + +- To filter issues by `status`, specify the `status` parameter in the URL. +`/-/integrations/jira/issues?status=In Progress` + +- To filter issues by `reporter`, specify a reporter's Jira display name for the +`author_username` parameter in the URL. `/-/integrations/jira/issues?author_username=John Smith` + +- To filter issues by `assignee`, specify their Jira display name for the +`assignee_username` parameter in the URL. `/-/integrations/jira/issues?assignee_username=John Smith` + ## Troubleshooting If these features do not work as expected, it is likely due to a problem with the way the integration settings were configured. diff --git a/doc/user/project/integrations/jira_cloud_configuration.md b/doc/user/project/integrations/jira_cloud_configuration.md index 619c94b282b..c7157b6bd0e 100644 --- a/doc/user/project/integrations/jira_cloud_configuration.md +++ b/doc/user/project/integrations/jira_cloud_configuration.md @@ -5,7 +5,7 @@ below to create one: 1. Log in to [`id.atlassian.com`](https://id.atlassian.com/manage-profile/security/api-tokens) with your email address. - NOTE: **Note** + NOTE: **Note:** It is important that the user associated with this email address has *write* access to projects in Jira. diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md index 1efd0ff3944..c8278a0f083 100644 --- a/doc/user/project/integrations/jira_server_configuration.md +++ b/doc/user/project/integrations/jira_server_configuration.md @@ -6,7 +6,7 @@ need to integrate with GitLab. As an example, we'll create a user named `gitlab` and add it to a new group named `gitlab-developers`. -NOTE: **Note** +NOTE: **Note:** It is important that the Jira user created for the integration is given 'write' access to your Jira projects. This is covered in the process below. diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 88668ab6c7d..79c55e2d140 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -14,8 +14,6 @@ want to configure.  -Below, you will find a list of the currently supported ones accompanied with comprehensive documentation. - ## Integrations listing Click on the service links to see further configuration instructions and details. @@ -28,6 +26,7 @@ Click on the service links to see further configuration instructions and details | Buildkite | Continuous integration and deployments | Yes | | [Bugzilla](bugzilla.md) | Bugzilla issue tracker | No | | Campfire | Simple web-based real-time group chat | No | +| [Confluence](../../../api/services.md#confluence-service) | Replaces the link to the internal wiki with a link to a Confluence Cloud Workspace | No | | Custom Issue Tracker | Custom issue tracker | No | | [Discord Notifications](discord_notifications.md) | Receive event notifications in Discord | No | | Drone CI | Continuous Integration platform built on Docker, written in Go | Yes | @@ -68,16 +67,16 @@ supported by `push_hooks` and `tag_push_hooks` events won't be executed. The number of branches or tags supported can be changed via [`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). -## Services templates +## Service templates -Services templates is a way to set some predefined values in the Service of -your liking which will then be pre-filled on each project's Service. +Service templates are a way to set predefined values for an integration across +all new projects on the instance. -Read more about [Services templates in this document](services_templates.md). +Read more about [Service templates in this document](services_templates.md). ## Troubleshooting integrations -Some integrations use service hooks for integration with external applications. To confirm which ones use service hooks, see the [integrations listing](#integrations-listing). GitLab stores details of service hook requests made within the last 2 days. To view details of the requests, go to the service's configuration page. +Some integrations use service hooks for integration with external applications. To confirm which ones use service hooks, see the [integrations listing](#integrations-listing) above. GitLab stores details of service hook requests made within the last 2 days. To view details of the requests, go to that integration's configuration page. The **Recent Deliveries** section lists the details of each request made within the last 2 days: @@ -88,17 +87,17 @@ The **Recent Deliveries** section lists the details of each request made within - Relative time in which the request was made To view more information about the request's execution, click the respective **View details** link. -On the details page, you can see the data sent by GitLab (request headers and body) and the data received by GitLab (response headers and body). +On the details page, you can see the request headers and body sent and received by GitLab. -From this page, you can repeat delivery with the same data by clicking **Resend Request**. +To repeat a delivery using the same data, click **Resend Request**.  ### Uninitialized repositories Some integrations fail with an error `Test Failed. Save Anyway` when you attempt to set them up on -uninitialized repositories. This is because the default service test uses push data to build the -payload for the test request, and it fails, because there are no push events for the project. +uninitialized repositories. Some integrations use push data to build the test payload, +and this error occurs when no push events exist in the project yet. To resolve this error, initialize the repository by pushing a test file to the project and set up the integration again. diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 0d17ff51372..f1567208a8f 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -19,7 +19,8 @@ There are two ways to set up Prometheus integration, depending on where your app - For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes). - For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus). -Once enabled, GitLab will automatically detect metrics from known services in the [metric library](#monitoring-cicd-environments). You can also [add your own metrics](#adding-custom-metrics). +Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics) and create +[custom dashboards](../../../operations/metrics/dashboards/index.md). ## Enabling Prometheus Integration @@ -32,11 +33,10 @@ GitLab can seamlessly deploy and manage Prometheus on a [connected Kubernetes cl #### Requirements - A [connected Kubernetes cluster](../clusters/index.md) -- Helm Tiller [installed by GitLab](../clusters/index.md#installing-applications) #### Getting started -Once you have a connected Kubernetes cluster with Helm installed, deploying a managed Prometheus is as easy as a single click. +Once you have a connected Kubernetes cluster, deploying a managed Prometheus is as easy as a single click. 1. Go to the **Operations > Kubernetes** page to view your connected clusters 1. Select the cluster you would like to deploy Prometheus to @@ -44,53 +44,6 @@ Once you have a connected Kubernetes cluster with Helm installed, deploying a ma  -#### Getting metrics to display on the Metrics Dashboard - -After completing the steps above, you will also need deployments in order to view the -**Operations > Metrics** page. Setting up [Auto DevOps](../../../topics/autodevops/index.md) -will help you to quickly create a deployment: - -1. Navigate to your project's **Operations > Kubernetes** page, and ensure that, - in addition to "Prometheus" and "Helm Tiller", you also have "Runner" and "Ingress" - installed. Once "Ingress" is installed, copy its endpoint. -1. Navigate to your project's **Settings > CI/CD** page. In the Auto DevOps section, - select a deployment strategy and save your changes. -1. On the same page, in the Variables section, add a variable named `KUBE_INGRESS_BASE_DOMAIN` - with the value of the Ingress endpoint you have copied in the previous step. Leave the type - as "Variable". -1. Navigate to your project's **CI/CD > Pipelines** page, and run a pipeline on any branch. -1. When the pipeline has run successfully, graphs will be available on the **Operations > Metrics** page. - - - -#### Using the Metrics Dashboard - -##### Select an environment - -The **Environment** dropdown box above the dashboard displays the list of all [environments](#monitoring-cicd-environments). -It enables you to search as you type through all environments and select the one you're looking for. - - - -##### Select a dashboard - -The **dashboard** dropdown box above the dashboard displays the list of all dashboards available for the project. -It enables you to search as you type through all dashboards and select the one you're looking for. - - - -##### Mark a dashboard as favorite - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0. - -When viewing a dashboard, click the empty **Star dashboard** **{star-o}** button to mark a -dashboard as a favorite. Starred dashboards display a solid star **{star}** button, -and appear at the top of the dashboard select list. - -To remove dashboard from the favorites list, click the solid **Unstar Dashboard** **{star}** button. - - - #### About managed Prometheus deployments Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). @@ -128,14 +81,24 @@ Installing and configuring Prometheus to monitor applications is fairly straight The actual configuration of Prometheus integration within GitLab is very simple. All you will need is the domain name or IP address of the Prometheus server you'd like -to integrate with. - -1. Navigate to the [Integrations page](overview.md#accessing-integrations). +to integrate with. If the Prometheus resource is secured with Google's Identity-Aware Proxy (IAP), +additional information like Client ID and Service Account credentials can be passed which +GitLab can use to access the resource. More information about authentication from a +service account can be found at Google's documentation for +[Authenticating from a service account](https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_service_account). + +1. Navigate to the [Integrations page](overview.md#accessing-integrations) at + **{settings}** **Settings > Integrations**. 1. Click the **Prometheus** service. -1. Provide the domain name or IP address of your server, for example `http://prometheus.example.com/` or `http://192.0.2.1/`. +1. For **API URL**, provide the domain name or IP address of your server, such as + `http://prometheus.example.com/` or `http://192.0.2.1/`. +1. (Optional) In **Google IAP Audience Client ID**, provide the Client ID of the + Prometheus OAuth Client secured with Google IAP. +1. (Optional) In **Google IAP Service Account JSON**, provide the contents of the + Service Account credentials file that is authorized to access the Prometheus resource. 1. Click **Save changes**. - + #### Thanos configuration in GitLab @@ -158,7 +121,7 @@ one of them will be used: [Prometheus manual configuration](#manual-configuration-of-prometheus) and a [managed Prometheus on Kubernetes](#managed-prometheus-on-kubernetes), the manual configuration takes precedence and is used to run queries from - [dashboards](#defining-custom-dashboards-per-project) and [custom metrics](#adding-custom-metrics). + [dashboards](../../../operations/metrics/dashboards/index.md#defining-custom-dashboards-per-project) and [custom metrics](../../../operations/metrics/index.md#adding-custom-metrics). - If you have managed Prometheus applications installed on Kubernetes clusters at **different** levels (project, group, instance), the order of precedence is described in [Cluster precedence](../../instance/clusters/index.md#cluster-precedence). @@ -166,884 +129,6 @@ one of them will be used: clusters at the **same** level, the Prometheus application of a cluster with a matching [environment scope](../../../ci/environments/index.md#scoping-environments-with-specs) is used. -## Monitoring CI/CD Environments - -Once configured, GitLab will attempt to retrieve performance metrics for any -environment which has had a successful deployment. - -GitLab will automatically scan the Prometheus server for metrics from known servers like Kubernetes and NGINX, and attempt to identify individual environments. The supported metrics and scan process is detailed in our [Prometheus Metrics Library documentation](prometheus_library/index.md). - -You can view the performance dashboard for an environment by [clicking on the monitoring button](../../../ci/environments/index.md#monitoring-environments). - -### Adding custom metrics - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28527) to [GitLab Core](https://about.gitlab.com/pricing/) 12.10. - -Custom metrics can be monitored by adding them on the monitoring dashboard page. Once saved, they will be displayed on the environment performance dashboard provided that either: - -- A [connected Kubernetes cluster](../clusters/add_remove_clusters.md) with the environment scope of `*` is used and [Prometheus installed on the cluster](#enabling-prometheus-integration) -- Prometheus is [manually configured](#manual-configuration-of-prometheus). - - - -A few fields are required: - -- **Name**: Chart title -- **Type**: Type of metric. Metrics of the same type will be shown together. -- **Query**: Valid [PromQL query](https://prometheus.io/docs/prometheus/latest/querying/basics/). -- **Y-axis label**: Y axis title to display on the dashboard. -- **Unit label**: Query units, for example `req / sec`. Shown next to the value. - -Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**, and **Y-axis label** match between metrics. For example, a metric with **Name** `Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display on the same chart as a second metric with the same values. A **Legend label** is suggested if this feature is used. - -#### Query Variables - -##### Predefined variables - -GitLab supports a limited set of [CI variables](../../../ci/variables/README.md) in the Prometheus query. This is particularly useful for identifying a specific environment, for example with `ci_environment_slug`. The supported variables are: - -- `ci_environment_slug` -- `kube_namespace` -- `ci_project_name` -- `ci_project_namespace` -- `ci_project_path` -- `ci_environment_name` -- `__range` - -NOTE: **Note:** -Variables for Prometheus queries must be lowercase. - -###### __range - -The `__range` variable is useful in Prometheus -[range vector selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#range-vector-selectors). -Its value is the total number of seconds in the dashboard's time range. -For example, if the dashboard time range is set to 8 hours, the value of -`__range` is `28800s`. - -##### User-defined variables - -[Variables can be defined](#templating-templating-properties) in a custom dashboard YAML file. - -##### Using variables - -Variables can be specified using double curly braces, such as `"{{ci_environment_slug}}"` ([added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20793) in GitLab 12.7). - -Support for the `"%{ci_environment_slug}"` format was -[removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31581) in GitLab 13.0. -Queries that continue to use the old format will show no data. - -#### Query Variables from URL - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214500) in GitLab 13.0. - -GitLab supports setting custom variables through URL parameters. Surround the variable -name with double curly braces (`{{example}}`) to interpolate the variable in a query: - -```plaintext -avg(sum(container_memory_usage_bytes{container_name!="{{pod}}"}) by (job)) without (job) /1024/1024/1024' -``` - -The URL for this query would be: - -```plaintext -http://gitlab.com/<user>/<project>/-/environments/<environment_id>/metrics?dashboard=.gitlab%2Fdashboards%2Fcustom.yml&pod=POD -``` - -#### Editing additional metrics from the dashboard - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9. - -You can edit existing additional custom metrics by clicking the **{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**. - - - -### Defining custom dashboards per project - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/59974) in GitLab 12.1. - -By default, all projects include a GitLab-defined Prometheus dashboard, which -includes a few key metrics, but you can also define your own custom dashboards. - -You may create a new file from scratch or duplicate a GitLab-defined Prometheus -dashboard. - -NOTE: **Note:** -The metrics as defined below do not support alerts, unlike -[custom metrics](#adding-custom-metrics). - -#### Adding a new dashboard to your project - -You can configure a custom dashboard by adding a new YAML file into your project's -`.gitlab/dashboards/` directory. In order for the dashboards to be displayed on -the project's **Operations > Metrics** page, the files must have a `.yml` -extension and should be present in the project's **default** branch. - -For example: - -1. Create `.gitlab/dashboards/prom_alerts.yml` under your repository's root - directory with the following contents: - - ```yaml - dashboard: 'Dashboard Title' - panel_groups: - - group: 'Group Title' - panels: - - type: area-chart - title: "Chart Title" - y_label: "Y-Axis" - y_axis: - format: number - precision: 0 - metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Instance: {{instance}}, method: {{method}}" - unit: "count" - ``` - - The above sample dashboard would display a single area chart. Each file should - define the layout of the dashboard and the Prometheus queries used to populate - data. - -1. Save the file, commit, and push to your repository. The file must be present in your **default** branch. -1. Navigate to your project's **Operations > Metrics** and choose the custom - dashboard from the dropdown. - -NOTE: **Note:** -Configuration files nested under subdirectories of `.gitlab/dashboards` are not -supported and will not be available in the UI. - -#### Duplicating a GitLab-defined dashboard - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37238) in GitLab 12.7. -> - From [GitLab 12.8 onwards](https://gitlab.com/gitlab-org/gitlab/-/issues/39505), custom metrics are also duplicated when you duplicate a dashboard. - -You can save a complete copy of a GitLab defined dashboard along with all custom metrics added to it. -Resulting `.yml` file can be customized and adapted to your project. -You can decide to save the dashboard `.yml` file in the project's **default** branch or in a -new branch. - -1. Click **Duplicate dashboard** in the dashboard dropdown. - - NOTE: **Note:** - You can duplicate only GitLab-defined dashboards. - -1. Enter the file name and other information, such as the new commit's message, and click **Duplicate**. - -If you select your **default** branch, the new dashboard becomes immediately available. -If you select another branch, this branch should be merged to your **default** branch first. - -#### Dashboard YAML syntax validation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33202) in GitLab 13.1. - -To confirm your dashboard definition contains valid YAML syntax: - -1. Navigate to **{doc-text}** **Repository > Files**. -1. Navigate to your dashboard file in your repository. -1. Review the information pane about the file, displayed above the file contents. - -Files with valid syntax display **Metrics Dashboard YAML definition is valid**, -and files with invalid syntax display **Metrics Dashboard YAML definition is invalid**. - - - -When **Metrics Dashboard YAML definition is invalid** at least one of the following messages is displayed: - -1. `dashboard: can't be blank` [learn more](#dashboard-top-level-properties) -1. `panel_groups: can't be blank` [learn more](#dashboard-top-level-properties) -1. `group: can't be blank` [learn more](#panel-group-panel_groups-properties) -1. `panels: can't be blank` [learn more](#panel-group-panel_groups-properties) -1. `metrics: can't be blank` [learn more](#panel-panels-properties) -1. `title: can't be blank` [learn more](#panel-panels-properties) -1. `query: can't be blank` [learn more](#metrics-metrics-properties) -1. `query_range: can't be blank` [learn more](#metrics-metrics-properties) -1. `unit: can't be blank` [learn more](#metrics-metrics-properties) -1. `YAML syntax: The parsed YAML is too big` - - This is displayed when the YAML file is larger than 1 MB. - -1. `YAML syntax: Invalid configuration format` - - This is displayed when the YAML file is empty or does not contain valid YAML. - -Metrics Dashboard YAML definition validation information is also available as a [GraphQL API field](../../../api/graphql/reference/index.md#metricsdashboard) - -#### Dashboard YAML properties - -Dashboards have several components: - -- Templating variables. -- Panel groups, which consist of panels. -- Panels, which support one or more metrics. - -The following tables outline the details of expected properties. - -##### **Dashboard (top-level) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `dashboard` | string | yes | Heading for the dashboard. Only one dashboard should be defined per file. | -| `panel_groups` | array | yes | The panel groups which should be on the dashboard. | -| `templating` | hash | no | Top level key under which templating related options can be added. | -| `links` | array | no | Add links to display on the dashboard. | - -##### **Templating (`templating`) properties** - -| Property | Type | Required | Description | -| -------- | ---- | -------- | ----------- | -| `variables` | hash | yes | Variables can be defined here. | - -Read the documentation on [templating](#templating-variables-for-metrics-dashboards). - -##### **Links (`links`) properties** - -| Property | Type | Required | Description | -| -------- | ---- | -------- | ----------- | -| `url` | string | yes | The address of the link. | -| `title` | string | no | Display title for the link. | -| `type` | string | no | Type of the link. Specifies the link type, can be: `grafana` | - -Read the documentation on [links](#add-related-links-to-custom-dashboards). - -##### **Panel group (`panel_groups`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `group` | string | required | Heading for the panel group. | -| `priority` | number | optional, defaults to order in file | Order to appear on the dashboard. Higher number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | -| `panels` | array | required | The panels which should be in the panel group. | - -##### **Panel (`panels`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------- | -| `type` | enum | no, defaults to `area-chart` | Specifies the chart type to use, can be: `area-chart`, `line-chart` or `anomaly-chart`. | -| `title` | string | yes | Heading for the panel. | -| `y_label` | string | no, but highly encouraged | Y-Axis label for the panel. | -| `y_axis` | string | no | Y-Axis configuration for the panel. | -| `max_value` | number | no | Denominator value used for calculating [percentile based results](#percentile-based-results) | -| `weight` | number | no, defaults to order in file | Order to appear within the grouping. Lower number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | -| `metrics` | array | yes | The metrics which should be displayed in the panel. Any number of metrics can be displayed when `type` is `area-chart` or `line-chart`, whereas only 3 can be displayed when `type` is `anomaly-chart`. | -| `links` | array | no | Add links to display on the chart's [context menu](#chart-context-menu). | - -##### **Axis (`panels[].y_axis`) properties** - -| Property | Type | Required | Description | -| ----------- | ------ | ----------------------------- | -------------------------------------------------------------------- | -| `name` | string | no, but highly encouraged | Y-Axis label for the panel. Replaces `y_label` if set. | -| `format` | string | no, defaults to `engineering` | Unit format used. See the [full list of units](prometheus_units.md). | -| `precision` | number | no, defaults to `2` | Number of decimal places to display in the number. | | - -##### **Metrics (`metrics`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). | -| `unit` | string | yes | Defines the unit of the query's return data. | -| `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. | -| `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | -| `query_range` | string | yes if `query` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query_range` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | -| `step` | number | no, value is calculated if not defined | Defines query resolution step width in float number of seconds. Metrics on the same panel should use the same `step` value. | - -##### Dynamic labels - -Dynamic labels are useful when multiple time series are returned from a Prometheus query. - -When a static label is used and a query returns multiple time series, then all the legend items will be labeled the same, which makes identifying each time series difficult: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Time Series" - unit: "count" -``` - -This may render a legend like this: - - - -For labels to be more explicit, using variables that reflect time series labels is a good practice. The variables will be replaced by the values of the time series labels when the legend is rendered: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Instance: {{instance}}, method: {{method}}" - unit: "count" -``` - -The resulting rendered legend will look like this: - - - -There is also a shorthand value for dynamic dashboard labels that make use of only one time series label: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Method" - unit: "count" -``` - -This works by lowercasing the value of `label` and, if there are more words separated by spaces, replacing those spaces with an underscore (`_`). The transformed value is then checked against the labels of the time series returned by the Prometheus query. If a time series label is found that is equal to the transformed value, then the label value will be used and rendered in the legend like this: - - - -#### Panel types for dashboards - -The below panel types are supported in monitoring dashboards. - -##### Area or Line Chart - -To add an area chart panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - type: area-chart # or line-chart - title: 'Area Chart Title' - y_label: "Y-Axis" - y_axis: - format: number - precision: 0 - metrics: - - id: area_http_requests_total - query_range: 'http_requests_total' - label: "Instance: {{instance}}, Method: {{method}}" - unit: "count" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | no | Type of panel to be rendered. Optional for area panel types | -| query_range | string | required | For area panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - - - -Starting in [version 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/202696), the y-axis values will automatically scale according to the data. Previously, it always started from 0. - -##### Anomaly chart - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16530) in GitLab 12.5. - -To add an anomaly chart panel type to a dashboard, add a panel with *exactly* 3 metrics. - -The first metric represents the current state, and the second and third metrics represent the upper and lower limit respectively: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - type: anomaly-chart - title: "Chart Title" - y_label: "Y-Axis" - metrics: - - id: anomaly_requests_normal - query_range: 'http_requests_total' - label: "# of Requests" - unit: "count" - metrics: - - id: anomaly_requests_upper_limit - query_range: 10000 - label: "Max # of requests" - unit: "count" - metrics: - - id: anomaly_requests_lower_limit - query_range: 2000 - label: "Min # of requests" - unit: "count" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | required | Must be `anomaly-chart` for anomaly panel types | -| query_range | yes | required | For anomaly panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) in every metric. | - - - -##### Bar chart - -To add a bar chart to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group title' - panels: - - type: bar - title: "Http Handlers" - x_label: 'Response Size' - y_axis: - name: "Handlers" - metrics: - - id: prometheus_http_response_size_bytes_bucket - query_range: "sum(increase(prometheus_http_response_size_bytes_bucket[1d])) by (handler)" - unit: 'Bytes' -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `type` | string | yes | Type of panel to be rendered. For bar chart types, set to `bar` | -| `query_range` | yes | yes | For bar chart, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) - - - -##### Column chart - -To add a column panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group title' - panels: - - title: "Column" - type: "column" - metrics: - - id: 1024_memory - query: 'avg(sum(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) by (job)) without (job) / count(avg(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) without (job)) /1024/1024' - unit: MB - label: "Memory Usage" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For column panel types, set to `column` | -| query_range | yes | yes | For column panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - - - -##### Stacked column - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30583) in GitLab 12.8. - -To add a stacked column panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard title' -priority: 1 -panel_groups: -- group: 'Group Title' - priority: 5 - panels: - - type: 'stacked-column' - title: "Stacked column" - y_label: "y label" - x_label: 'x label' - metrics: - - id: memory_1 - query_range: 'memory_query' - label: "memory query 1" - unit: "count" - series_name: 'group 1' - - id: memory_2 - query_range: 'memory_query_2' - label: "memory query 2" - unit: "count" - series_name: 'group 2' - -``` - - - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `type` | string | yes | Type of panel to be rendered. For stacked column panel types, set to `stacked-column` | -| `query_range` | yes | yes | For stacked column panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - -##### Single Stat - -To add a single stat panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Single Stat" - type: "single-stat" - metrics: - - id: 10 - query: 'max(go_memstats_alloc_bytes{job="prometheus"})' - unit: MB - label: "Total" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For single stat panel types, set to `single-stat` | -| query | string | yes | For single stat panel types, you must use an [instant query](https://prometheus.io/docs/prometheus/latest/querying/api/#instant-queries) | - - - -###### Percentile based results - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201946) in GitLab 12.8. - -Query results sometimes need to be represented as a percentage value out of 100. You can use the `max_value` property at the root of the panel definition: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Single Stat" - type: "single-stat" - max_value: 100 - metrics: - - id: 10 - query: 'max(go_memstats_alloc_bytes{job="prometheus"})' - unit: '%' - label: "Total" -``` - -For example, if you have a query value of `53.6`, adding `%` as the unit results in a single stat value of `53.6%`, but if the maximum expected value of the query is `120`, the value would be `44.6%`. Adding the `max_value` causes the correct percentage value to display. - -##### Heatmaps - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30581) in GitLab 12.5. - -To add a heatmap panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Heatmap" - type: "heatmap" - metrics: - - id: 10 - query: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[60m])) by (status_code)' - unit: req/sec - label: "Status code" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For heatmap panel types, set to `heatmap` | -| query_range | yes | yes | For area panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - - - -### Templating variables for metrics dashboards - -Templating variables can be used to make your metrics dashboard more versatile. - -#### Templating variable types - -`templating` is a top-level key in the -[dashboard YAML](#dashboard-top-level-properties). -Define your variables in the `variables` key, under `templating`. The value of -the `variables` key should be a hash, and each key under `variables` -defines a templating variable on the dashboard, and may contain alphanumeric and underscore characters. - -A variable can be used in a Prometheus query in the same dashboard using the syntax -described [here](#using-variables). - -##### `text` variable type - -CAUTION: **Warning:** -This variable type is an _alpha_ feature, and is subject to change at any time -without prior notice! - -For each `text` variable defined in the dashboard YAML, there will be a free text -box on the dashboard UI, allowing you to enter a value for each variable. - -The `text` variable type supports a simple and a full syntax. - -###### Simple syntax - -This example creates a variable called `variable1`, with a default value -of `default value`: - -```yaml -templating: - variables: - variable1: 'default value' # `text` type variable with `default value` as its default. -``` - -###### Full syntax - -This example creates a variable called `variable1`, with a default value of `default`. -The label for the text box on the UI will be the value of the `label` key: - -```yaml -templating: - variables: - variable1: # The variable name that can be used in queries. - label: 'Variable 1' # (Optional) label that will appear in the UI for this text box. - type: text - options: - default_value: 'default' # (Optional) default value. -``` - -##### `custom` variable type - -CAUTION: **Warning:** -This variable type is an _alpha_ feature, and is subject to change at any time -without prior notice! - -Each `custom` variable defined in the dashboard YAML creates a dropdown -selector on the dashboard UI, allowing you to select a value for each variable. - -The `custom` variable type supports a simple and a full syntax. - -###### Simple syntax - -This example creates a variable called `variable1`, with a default value of `value1`. -The dashboard UI will display a dropdown with `value1`, `value2` and `value3` -as the choices. - -```yaml -templating: - variables: - variable1: ['value1', 'value2', 'value3'] -``` - -###### Full syntax - -This example creates a variable called `variable1`, with a default value of `value_option_2`. -The label for the text box on the UI will be the value of the `label` key. -The dashboard UI will display a dropdown with `Option 1` and `Option 2` -as the choices. - -If you select `Option 1` from the dropdown, the variable will be replaced with `value option 1`. -Similarly, if you select `Option 2`, the variable will be replaced with `value_option_2`: - -```yaml -templating: - variables: - variable1: # The variable name that can be used in queries. - label: 'Variable 1' # (Optional) label that will appear in the UI for this dropdown. - type: custom - options: - values: - - value: 'value option 1' # The value that will replace the variable in queries. - text: 'Option 1' # (Optional) Text that will appear in the UI dropdown. - - value: 'value_option_2' - text: 'Option 2' - default: true # (Optional) This option should be the default value of this variable. -``` - -### Add related links to custom dashboards - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216385) in GitLab 13.1. - -You can embed links to other dashboards or external services in your custom -dashboard by adding **Related links** to your dashboard's YAML file. Related links -open in the same tab as the dashboard. Related links can be displayed in the -following locations on your dashboard: - -- At the top of your dashboard as the top level [`links` dashboard property](#dashboard-top-level-properties). -- In charts context menus as the [`links` property of a panel](#panel-panels-properties). - -Related links can contain the following attributes: - -- `url`: The full URL to the link. Required. -- `title`: A phrase describing the link. Optional. If this attribute is not set, - the full URL is used for the link title. -- `type`: A string declaring the type of link. Optional. If set to `grafana`, the - dashboard's time range values are converted to Grafana's time range format and - appended to the `url`. - -The dashboard's time range is appended to the `url` as URL parameters. - -The following example shows two related links (`GitLab.com` and `GitLab Documentation`) -added to a dashboard: - - - -#### Links Syntax - -```yaml -links: - - title: GitLab.com - url: https://gitlab.com - - title: GitLab Documentation - url: https://docs.gitlab.com - - title: Public Grafana playground dashboard - url: https://play.grafana.org/d/000000012/grafana-play-home?orgId=1 - type: grafana -``` - -### View and edit the source file of a custom dashboard - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34779) in GitLab 12.5. - -When viewing a custom dashboard of a project, you can view the original -`.yml` file by clicking on the **Edit dashboard** button. - -### Chart Context Menu - -From each of the panels in the dashboard, you can access the context menu by clicking the **{ellipsis_v}** **More actions** dropdown box above the upper right corner of the panel to take actions related to the chart's data. - - - -The options are: - -- [Expand panel](#expand-panel) -- [View logs](#view-logs-ultimate) -- [Download CSV](#downloading-data-as-csv) -- [Copy link to chart](#embedding-gitlab-managed-kubernetes-metrics) -- [Alerts](#setting-up-alerts-for-prometheus-metrics) - -### Dashboard Annotations - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`). -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/215224) in GitLab 13.0. - -You can use **Metrics Dashboard Annotations** to mark any important events on -every metrics dashboard by adding annotations to it. While viewing a dashboard, -annotation entries assigned to the selected time range will be automatically -fetched and displayed on every chart within that dashboard. On mouse hover, each -annotation presents additional details, including the exact time of an event and -its description. - -You can create annotations by making requests to the -[Metrics dashboard annotations API](../../../api/metrics_dashboard_annotations.md) - - - -#### Retention policy - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211433) in GitLab 13.01. - -To avoid excessive storage space consumption by stale annotations, records attached -to time periods older than two weeks are removed daily. This recurring background -job runs at 1:00 a.m. local server time. - -### Expand panel - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3100) in GitLab 13.0. - -To view a larger version of a visualization, expand the panel by clicking the -**{ellipsis_v}** **More actions** icon and selecting **Expand panel**. - -To return to the metrics dashboard, click the **Back** button in your -browser, or pressing the <kbd>Esc</kbd> key. - -### View Logs **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/122013) in GitLab 12.8. - -If you have [Logs](../clusters/kubernetes_pod_logs.md) enabled, -you can navigate from the charts in the dashboard to view Logs by -clicking on the context menu in the upper-right corner. - -If you use the **Timeline zoom** function at the bottom of the chart, logs will narrow down to the time range you selected. - -### Timeline zoom and URL sharing - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198910) in GitLab 12.8. - -You can use the **Timeline zoom** function at the bottom of a chart to zoom in -on a date and time of your choice. When you click and drag the sliders to select -a different beginning or end date of data to display, GitLab adds your selected start -and end times to the URL, enabling you to share specific timeframes more easily. - -### Downloading data as CSV - -Data from Prometheus charts on the metrics dashboard can be downloaded as CSV. - -### Setting up alerts for Prometheus metrics - -#### Managed Prometheus instances - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6590) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2 for [custom metrics](#adding-custom-metrics), and 11.3 for [library metrics](prometheus_library/metrics.md). - -For managed Prometheus instances using auto configuration, alerts for metrics [can be configured](#adding-custom-metrics) directly in the performance dashboard. - -To set an alert: - -1. Click on the ellipsis icon in the top right corner of the metric you want to create the alert for. -1. Choose **Alerts** -1. Set threshold and operator. -1. Click **Add** to save and activate the alert. - - - -To remove the alert, click back on the alert icon for the desired metric, and click **Delete**. - -#### External Prometheus instances - ->- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8. ->- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10. - -For manually configured Prometheus servers, a notify endpoint is provided to use with Prometheus webhooks. If you have manual configuration enabled, an **Alerts** section is added to **Settings > Integrations > Prometheus**. This contains the *URL* and *Authorization Key*. The **Reset Key** button will invalidate the key and generate a new one. - - - -To send GitLab alert notifications, copy the *URL* and *Authorization Key* into the [`webhook_configs`](https://prometheus.io/docs/alerting/configuration/#webhook_config) section of your Prometheus Alertmanager configuration: - -```yaml -receivers: - name: gitlab - webhook_configs: - - http_config: - bearer_token: 9e1cbfcd546896a9ea8be557caf13a76 - send_resolved: true - url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json - ... -``` - -In order for GitLab to associate your alerts with an [environment](../../../ci/environments/index.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab. - -NOTE: **Note** -In GitLab versions 13.1 and greater, you can configure your manually configured Prometheus server to use the [Generic alerts integration](generic_alerts.md). - -### Taking action on incidents **(ULTIMATE)** - ->- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11. ->- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue. - -Alerts can be used to trigger actions, like opening an issue automatically (disabled by default since `13.1`). To configure the actions: - -1. Navigate to your project's **Settings > Operations > Incidents**. -1. Enable the option to create issues. -1. Choose the [issue template](../description_templates.md) to create the issue from. -1. Optionally, select whether to send an email notification to the developers of the project. -1. Click **Save changes**. - -Once enabled, an issue will be opened automatically when an alert is triggered which contains values extracted from [alert's payload](https://prometheus.io/docs/alerting/configuration/#webhook_config -): - -- Issue author: `GitLab Alert Bot` -- Issue title: Extract from `annotations/title`, `annotations/summary` or `labels/alertname` -- Alert `Summary`: A list of properties - - `starts_at`: Alert start time via `startsAt` - - `full_query`: Alert query extracted from `generatorURL` - - Optional list of attached annotations extracted from `annotations/*` -- Alert [GFM](../../markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown` - -When GitLab receives a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicating that it was closed automatically by the GitLab Alert bot. - -To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field. - -Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well. - -If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../permissions.md#project-members-permissions) of the project. - ## Determining the performance impact of a merge > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10408) in GitLab 9.2. @@ -1069,174 +154,3 @@ Performance data will be available for the duration it is persisted on the Prometheus server.  - -## Embedding metric charts within GitLab Flavored Markdown - -### Embedding GitLab-managed Kubernetes metrics - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/29691) in GitLab 12.2. - -It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm) fields such as issue or merge request descriptions. The maximum number of embedded charts allowed in a GitLab Flavored Markdown field is 100. - -This can be useful if you are sharing an application incident or performance -metrics to others and want to have relevant information directly available. - -NOTE: **Note:** -Requires [Kubernetes](prometheus_library/kubernetes.md) metrics. - -To display metric charts, include a link of the form `https://<root_url>/<project>/-/environments/<environment_id>/metrics`: - - - -GitLab unfurls the link as an embedded metrics panel: - - - -You can also embed a single chart. To get a link to a chart, click the -**{ellipsis_v}** **More actions** menu in the upper right corner of the chart, -and select **Copy link to chart**, as shown in this example: - - - -The following requirements must be met for the metric to unfurl: - -- The `<environment_id>` must correspond to a real environment. -- Prometheus must be monitoring the environment. -- The GitLab instance must be configured to receive data from the environment. -- The user must be allowed access to the monitoring dashboard for the environment ([Reporter or higher](../../permissions.md)). -- The dashboard must have data within the last 8 hours. - - If all of the above are true, then the metric will unfurl as seen below: - - - -Metric charts may also be hidden: - - - -You can open the link directly into your browser for a [detailed view of the data](#expand-panel). - -### Embedding metrics in issue templates - -It is also possible to embed either the default dashboard metrics or individual metrics in issue templates. For charts to render side-by-side, links to the entire metrics dashboard or individual metrics should be separated by either a comma or a space. - - - -### Embedding metrics based on alerts in incident issues - -For [GitLab-managed alerting rules](#setting-up-alerts-for-prometheus-metrics), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after. - -For [manually configured Prometheus instances](#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met: - -- The alert corresponds to an environment managed through GitLab. -- The alert corresponds to a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries). -- The alert contains the required attributes listed in the chart below. - -| Attributes | Required | Description | -| ---------- | -------- | ----------- | -| `annotations/gitlab_environment_name` | Yes | Name of the GitLab-managed environment corresponding to the alert | -| One of `annotations/title`, `annotations/summary`, `labels/alertname` | Yes | Will be used as the chart title | -| `annotations/gitlab_y_label` | No | Will be used as the chart's y-axis label | - -### Embedding Cluster Health Charts **(ULTIMATE)** - -> [Introduced](<https://gitlab.com/gitlab-org/gitlab/-/issues/40997>) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. - -[Cluster Health Metrics](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) can also be embedded in [GitLab-flavored Markdown](../../markdown.md). - -To embed a metric chart, include a link to that chart in the form `https://<root_url>/<project>/-/cluster/<cluster_id>?<query_params>` anywhere that GitLab-flavored Markdown is supported. To generate and copy a link to the chart, follow the instructions in the [Cluster Health Metric documentation](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate). - -The following requirements must be met for the metric to unfurl: - -- The `<cluster_id>` must correspond to a real cluster. -- Prometheus must be monitoring the cluster. -- The user must be allowed access to the project cluster metrics. -- The dashboards must be reporting data on the [Cluster Health Page](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) - - If the above requirements are met, then the metric will unfurl as seen below. - - - -### Embedding Grafana charts - -Grafana metrics can be embedded in [GitLab Flavored Markdown](../../markdown.md). - -#### Embedding charts via Grafana Rendered Images - -It is possible to embed live [Grafana](https://docs.gitlab.com/omnibus/settings/grafana.html) charts in issues, as a [direct linked rendered image](https://grafana.com/docs/grafana/latest/reference/share_panel/#direct-link-rendered-image). - -The sharing dialog within Grafana provides the link, as highlighted below. - - - -NOTE: **Note:** -For this embed to display correctly, the Grafana instance must be available to the target user, either as a public dashboard, or on the same network. - -Copy the link and add an image tag as [inline HTML](../../markdown.md#inline-html) in your Markdown. You may tweak the query parameters as required. For instance, removing the `&from=` and `&to=` parameters will give you a live chart. Here is example markup for a live chart from GitLab's public dashboard: - -```html -<img src="https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?orgId=1&refresh=30s&var-env=gprd&var-environment=gprd&var-prometheus=prometheus-01-inf-gprd&var-prometheus_app=prometheus-app-01-inf-gprd&var-backend=All&var-type=All&var-stage=main&from=1580444107655&to=1580465707655"/> -``` - -This will render like so: - - - -#### Embedding charts via integration with Grafana HTTP API - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31376) in GitLab 12.5. - -Each project can support integration with one Grafana instance. This configuration allows a user to copy a link to a panel in Grafana, then paste it into a GitLab Markdown field. The chart will be rendered in the GitLab chart format. - -Prerequisites for embedding from a Grafana instance: - -1. The datasource must be a Prometheus instance. -1. The datasource must be proxyable, so the HTTP Access setting should be set to `Server`. - - - -##### Setting up the Grafana integration - -1. [Generate an Admin-level API Token in Grafana.](https://grafana.com/docs/grafana/latest/http_api/auth/#create-api-token) -1. In your GitLab project, navigate to **Settings > Operations > Grafana Authentication**. -1. To enable the integration, check the "Active" checkbox. -1. For "Grafana URL", enter the base URL of the Grafana instance. -1. For "API Token", enter the Admin API Token you just generated. -1. Click **Save Changes**. - -##### Generating a link to a chart - -1. In Grafana, navigate to the dashboard you wish to embed a panel from. -  -1. In the upper-left corner of the page, select a specific value for each variable required for the queries in the chart. -  -1. In Grafana, click on a panel's title, then click **Share** to open the panel's sharing dialog to the **Link** tab. If you click the _dashboard's_ share panel instead, GitLab will attempt to embed the first supported panel on the dashboard (if available). -1. If your Prometheus queries use Grafana's custom template variables, ensure that the "Template variables" option is toggled to **On**. Of Grafana global template variables, only `$__interval`, `$__from`, and `$__to` are currently supported. Toggle **On** the "Current time range" option to specify the time range of the chart. Otherwise, the default range will be the last 8 hours. -  -1. Click **Copy** to copy the URL to the clipboard. -1. In GitLab, paste the URL into a Markdown field and save. The chart will take a few moments to render. -  - -## Metrics dashboard visibility - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201924) in GitLab 13.0. - -You can set the visibility of the metrics dashboard to **Only Project Members** -or **Everyone With Access**. When set to **Everyone with Access**, the metrics -dashboard is visible to authenticated and non-authenticated users. - -## Troubleshooting - -When troubleshooting issues with a managed Prometheus app, it is often useful to -[view the Prometheus UI](../../../development/prometheus.md#access-the-ui-of-a-prometheus-managed-application-in-kubernetes). - -### "No data found" error on Metrics dashboard page - -If the "No data found" screen continues to appear, it could be due to: - -- No successful deployments have occurred to this environment. -- Prometheus does not have performance data for this environment, or the metrics - are not labeled correctly. To test this, connect to the Prometheus server and - [run a query](prometheus_library/kubernetes.md#metrics-supported), replacing `$CI_ENVIRONMENT_SLUG` - with the name of your environment. -- You may need to re-add the GitLab predefined common metrics. This can be done by running the [import common metrics Rake task](../../../administration/raketasks/maintenance.md#import-common-metrics). diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index b2bc217e8bf..7bebe7b1e65 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -8,7 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22133) in GitLab 11.7. -NOTE: **Note:** NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. +NOTE: **Note:** +NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the built-in Prometheus metrics included with Kubernetes NGINX Ingress controller [version 0.16.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0160) onward. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md index 6ba0a7610f6..326931e9790 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md @@ -8,7 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13438) in GitLab 9.5. -NOTE: **Note:** [NGINX Ingress version 0.16](nginx_ingress.md) and above have built-in Prometheus metrics, which are different than the VTS based metrics. +NOTE: **Note:** +[NGINX Ingress version 0.16](nginx_ingress.md) and above have built-in Prometheus metrics, which are different than the VTS based metrics. GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the included VTS Prometheus metrics exporter in [version 0.9.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#09-beta1) through [0.15.x](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0150). diff --git a/doc/user/project/integrations/prometheus_units.md b/doc/user/project/integrations/prometheus_units.md index 7b216ced1cc..ee4f3ed77d4 100644 --- a/doc/user/project/integrations/prometheus_units.md +++ b/doc/user/project/integrations/prometheus_units.md @@ -1,175 +1,5 @@ --- -stage: Monitor -group: APM -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../../../operations/metrics/dashboards/yaml_number_format.md' --- -# Unit formats reference - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201999) in GitLab 12.9. - -You can select units to format your charts by adding `format` to your -[axis configuration](prometheus.md#dashboard-yaml-properties). - -## Internationalization and localization - -Currently, your [internationalization and localization options](https://en.wikipedia.org/wiki/Internationalization_and_localization) for number formatting are dependent on the system you are using i.e. your OS or browser. - -## Engineering Notation - -For generic or default data, numbers are formatted according to the current locale in [engineering notation](https://en.wikipedia.org/wiki/Engineering_notation). - -While an [engineering notation exists for the web](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat), GitLab uses a version based off the [scientific notation](https://en.wikipedia.org/wiki/Scientific_notation). GitLab formatting acts in accordance with SI prefixes. For example, using GitLab notation, `1500.00` becomes `1.5k` instead of `1.5E3`. Keep this distinction in mind when using the engineering notation for your metrics. - -Formats: `engineering` - -SI prefixes: - -| Name | Symbol | Value | -| ---------- | ------- | -------------------------- | -| `yotta` | Y | 1000000000000000000000000 | -| `zetta` | Z | 1000000000000000000000 | -| `exa` | E | 1000000000000000000 | -| `peta` | P | 1000000000000000 | -| `tera` | T | 1000000000000 | -| `giga` | G | 1000000000 | -| `mega` | M | 1000000 | -| `kilo` | k | 1000 | -| `milli` | m | 0.001 | -| `micro` | μ | 0.000001 | -| `nano` | n | 0.000000001 | -| `pico` | p | 0.000000000001 | -| `femto` | f | 0.000000000000001 | -| `atto` | a | 0.000000000000000001 | -| `zepto` | z | 0.000000000000000000001 | -| `yocto` | y | 0.000000000000000000000001 | - -**Examples:** - -| Data | Displayed | -| --------------------------------- | --------- | -| `0.000000000000000000000008` | 8y | -| `0.000000000000000000008` | 8z | -| `0.000000000000000008` | 8a | -| `0.000000000000008` | 8f | -| `0.000000000008` | 8p | -| `0.000000008` | 8n | -| `0.000008` | 8μ | -| `0.008` | 8m | -| `10` | 10 | -| `1080` | 1.08k | -| `18000` | 18k | -| `18888` | 18.9k | -| `188888` | 189k | -| `18888888` | 18.9M | -| `1888888888` | 1.89G | -| `1888888888888` | 1.89T | -| `1888888888888888` | 1.89P | -| `1888888888888888888` | 1.89E | -| `1888888888888888888888` | 1.89Z | -| `1888888888888888888888888` | 1.89Y | -| `1888888888888888888888888888` | 1.89e+27 | - -## Numbers - -For number data, numbers are formatted according to the current locale. - -Formats: `number` - -**Examples:** - -| Data | Displayed | -| ---------- | --------- | -| `10` | 1 | -| `1000` | 1,000 | -| `1000000` | 1,000,000 | - -## Percentage - -For percentage data, format numbers in the chart with a `%` symbol. - -Formats supported: `percent`, `percentHundred` - -**Examples:** - -| Format | Data | Displayed | -| ---------------- | ----- | --------- | -| `percent` | `0.5` | 50% | -| `percent` | `1` | 100% | -| `percent` | `2` | 200% | -| `percentHundred` | `50` | 50% | -| `percentHundred` | `100` | 100% | -| `percentHundred` | `200` | 200% | - -## Duration - -For time durations, format numbers in the chart with a time unit symbol. - -Formats supported: `milliseconds`, `seconds` - -**Examples:** - -| Format | Data | Displayed | -| -------------- | ------ | --------- | -| `milliseconds` | `10` | 10ms | -| `milliseconds` | `500` | 100ms | -| `milliseconds` | `1000` | 1000ms | -| `seconds` | `10` | 10s | -| `seconds` | `500` | 500s | -| `seconds` | `1000` | 1000s | - -## Digital (Metric) - -Converts a number of bytes using metric prefixes. It scales to -use the unit that's the best fit. - -Formats supported: - -- `decimalBytes` -- `kilobytes` -- `megabytes` -- `gigabytes` -- `terabytes` -- `petabytes` - -**Examples:** - -| Format | Data | Displayed | -| -------------- | --------- | --------- | -| `decimalBytes` | `1` | 1B | -| `decimalBytes` | `1000` | 1kB | -| `decimalBytes` | `1000000` | 1MB | -| `kilobytes` | `1` | 1kB | -| `kilobytes` | `1000` | 1MB | -| `kilobytes` | `1000000` | 1GB | -| `megabytes` | `1` | 1MB | -| `megabytes` | `1000` | 1GB | -| `megabytes` | `1000000` | 1TB | - -## Digital (IEC) - -Converts a number of bytes using binary prefixes. It scales to -use the unit that's the best fit. - -Formats supported: - -- `bytes` -- `kibibytes` -- `mebibytes` -- `gibibytes` -- `tebibytes` -- `pebibytes` - -**Examples:** - -| Format | Data | Displayed | -| ----------- | ------------- | --------- | -| `bytes` | `1` | 1B | -| `bytes` | `1024` | 1KiB | -| `bytes` | `1024 * 1024` | 1MiB | -| `kibibytes` | `1` | 1KiB | -| `kibibytes` | `1024` | 1MiB | -| `kibibytes` | `1024 * 1024` | 1GiB | -| `mebibytes` | `1` | 1MiB | -| `mebibytes` | `1024` | 1GiB | -| `mebibytes` | `1024 * 1024` | 1TiB | +This document was moved to [another location](../../../operations/metrics/dashboards/yaml_number_format.md). diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 9e1cdf0490c..c92ddf38ad2 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -7,7 +7,6 @@ | Field | Description | | ----- | ----------- | - | `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | | `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project. **This is currently not being used and will be removed in a future release.** | diff --git a/doc/user/project/integrations/services_templates.md b/doc/user/project/integrations/services_templates.md index 8a88df88629..bc2bdde2f64 100644 --- a/doc/user/project/integrations/services_templates.md +++ b/doc/user/project/integrations/services_templates.md @@ -1,28 +1,25 @@ -# Services templates +# Service templates -A GitLab administrator can add a service template that sets a default for each -project. After a service template is enabled, it will be applied to **all** -projects that don't have it already enabled and its details will be pre-filled -on the project's Service page. By disabling the template, it will be disabled -for new projects only. +Using a service template, GitLab administrators can provide default values for configuring integrations at the project level. + +When you enable a service template, the defaults are applied to **all** projects that do not +already have the integration enabled or do not otherwise have custom values saved. +The values are pre-filled on each project's configuration page for the applicable integration. + +If you disable the template, these values no longer appear as defaults, while +any values already saved for an integration remain unchanged. ## Enable a service template Navigate to the **Admin Area > Service Templates** and choose the service template you wish to create. -## Services for external issue trackers +## Service for external issue trackers -In the image below you can see how a service template for Redmine would look -like. +The following image shows an example service template for Redmine.  ---- - For each project, you will still need to configure the issue tracking URLs by replacing `:issues_tracker_id` in the above screenshot with the ID used -by your external issue tracker. Prior to GitLab v7.8, this ID was configured in -the project settings, and GitLab would automatically update the URL configured -in `gitlab.yml`. This behavior is now deprecated and all issue tracker URLs -must be configured directly within the project's **Integrations** settings. +by your external issue tracker. diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 79fc8eceddf..6c5dc787c5e 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -1,25 +1,45 @@ # Slack Notifications Service -The Slack Notifications Service allows your GitLab project to send events (e.g. issue created) to your existing Slack team as notifications. This requires configurations in both Slack and GitLab. +The Slack Notifications Service allows your GitLab project to send events +(such as issue creation) to your existing Slack team as notifications. Setting up +Slack notifications requires configuration changes for both Slack and GitLab. -> Note: You can also use Slack slash commands to control GitLab inside Slack. This is the separately configured [Slack slash commands](slack_slash_commands.md). +NOTE: **Note:** +You can also use Slack slash commands to control GitLab inside Slack. This is the +separately configured [Slack slash commands](slack_slash_commands.md). -## Slack Configuration +## Slack configuration 1. Sign in to your Slack team and [start a new Incoming WebHooks configuration](https://my.slack.com/services/new/incoming-webhook). -1. Select the Slack channel where notifications will be sent to by default. Click the **Add Incoming WebHooks integration** button to add the configuration. -1. Copy the **Webhook URL**, which we'll use later in the GitLab configuration. +1. Select the Slack channel where notifications will be sent to by default. + Click the **Add Incoming WebHooks integration** button to add the configuration. +1. Copy the **Webhook URL**, which we will use later in the GitLab configuration. -## GitLab Configuration +## GitLab configuration -1. Navigate to the [Integrations page](overview.md#accessing-integrations) in your project's settings, i.e. **Project > Settings > Integrations**. +1. Open your project's page, and navigate to your project's + [Integrations page](overview.md#accessing-integrations) at + **{settings}** **Settings > Integrations**. 1. Select the **Slack notifications** integration to configure it. -1. Ensure that the **Active** toggle is enabled. -1. Check the checkboxes corresponding to the GitLab events you want to send to Slack as a notification. -1. For each event, optionally enter the Slack channel names where you want to send the event, separated by a comma. If left empty, the event will be sent to the default channel that you configured in the Slack Configuration step. **Note:** Usernames and private channels are not supported. To send direct messages, use the Member ID found under user's Slack profile. -1. Paste the **Webhook URL** that you copied from the Slack Configuration step. -1. Optionally customize the Slack bot username that will be sending the notifications. -1. Configure the remaining options and click `Save changes`. +1. Click **Enable integration**. +1. In **Trigger**, select the checkboxes for each type of GitLab event to send to Slack as a + notification. See [Triggers available for Slack notifications](#triggers-available-for-slack-notifications) + for a full list. By default, messages are sent to the channel you configured during + [Slack integration](#slack-configuration). +1. (Optional) To send messages to a different channel, multiple channels, or as a direct message: + - To send messages to channels, enter the Slack channel names, separated by commas. + - To send direct messages, use the Member ID found in the user's Slack profile. + + NOTE: **Note:** + Usernames and private channels are not supported. + +1. In **Webhook**, provide the webhook URL that you copied from the + [Slack integration](#slack-configuration) step. +1. (Optional) In **Username**, provide the username of the Slack bot that sends the notifications. +1. Select the **Notify only broken pipelines** check box to only notify on failures. +1. In the **Branches to be notified** select box, choose which types of branches + to send notifications for. +1. Click **Test settings and save changes**. Your Slack team will now start receiving GitLab event notifications as configured. @@ -43,14 +63,14 @@ The following triggers are available for Slack notifications: ## Troubleshooting -If you're having trouble with the Slack integration not working, then start by +If your Slack integration is not working, start troubleshooting by searching through the [Sidekiq logs](../../../administration/logs.md#sidekiqlog) for errors relating to your Slack service. ### Something went wrong on our end -This is a generic error shown in the GitLab UI and doesn't mean much by itself. -You'll need to look in [the logs](../../../administration/logs.md#productionlog) to find +This is a generic error shown in the GitLab UI and does not mean much by itself. +Review [the logs](../../../administration/logs.md#productionlog) to find an error message and keep troubleshooting from there. ### `certificate verify failed` @@ -83,10 +103,10 @@ result = Net::HTTP.get(URI('https://<SLACK URL>'));0 result = Net::HTTP.get(URI('https://<GITLAB URL>'));0 ``` -If it's an issue with GitLab not trusting HTTPS connections to itself, then you may simply +If GitLab is not trusting HTTPS connections to itself, then you may need to [add your certificate to GitLab's trusted certificates](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). -If it's an issue with GitLab not trusting connections to Slack, then the GitLab -OpenSSL trust store probably got messed up somehow. Typically this is from overriding -the trust store with `gitlab_rails['env'] = {"SSL_CERT_FILE" => "/path/to/file.pem"}` +If GitLab is not trusting connections to Slack, then the GitLab +OpenSSL trust store is incorrect. Some typical causes: overriding +the trust store with `gitlab_rails['env'] = {"SSL_CERT_FILE" => "/path/to/file.pem"}`, or by accidentally modifying the default CA bundle `/opt/gitlab/embedded/ssl/certs/cacert.pem`. diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index 119a53f5c35..e067ab6071e 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -13,7 +13,6 @@ To enable YouTrack integration in a project: | Field | Description | |:----------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| - | **Description** | Name for the issue tracker (to differentiate between instances, for example). | | **Project URL** | URL to the project in YouTrack which is being linked to this GitLab project. | | **Issues URL** | URL to the issue in YouTrack project that is linked to this GitLab project. Note that the **Issues URL** requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 89b17609698..1831563332f 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -42,8 +42,6 @@ Check all the [GitLab Enterprise features for issue boards](#gitlab-enterprise-f  ---- - ## How it works The Issue Board feature builds on GitLab's existing @@ -98,8 +96,6 @@ If you have the labels "**backend**", "**frontend**", "**staging**", and  ---- - ### Use cases for multiple issue boards With [multiple issue boards](#multiple-issue-boards), @@ -252,8 +248,6 @@ clicking **View scope**.  ---- - ### Focus mode > - [Introduced]((https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#issue-boards-focus-mode-ees-eep)) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1. @@ -275,8 +269,6 @@ especially in combination with [assignee lists](#assignee-lists-premium).  ---- - ### Group issue boards **(PREMIUM)** > [Introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. @@ -292,8 +284,6 @@ Multiple group issue boards were originally [introduced](https://about.gitlab.co  ---- - ### Assignee lists **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. @@ -313,8 +303,6 @@ To remove an assignee list, just as with a label list, click the trash icon.  ---- - ### Milestone lists **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6469) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.2. @@ -332,8 +320,6 @@ As in other list types, click the trash icon to remove a list.  ---- - ## Work In Progress limits **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11403) in GitLab 12.7 @@ -365,8 +351,6 @@ status.  ---- - ## Actions you can take on an issue board - [Create a new list](#create-a-new-list). @@ -437,8 +421,6 @@ the list by filtering by author, assignee, milestone, and label.  ---- - ### Remove an issue from a list Removing an issue from a list can be done by clicking the issue card and then @@ -447,8 +429,6 @@ respective label is removed.  ---- - ### Filter issues You should be able to use the filters on top of your issue board to show only @@ -492,8 +472,6 @@ to another list the label changes and a system not is recorded.  ---- - ### Drag issues between lists When dragging issues between lists, different behavior occurs depending on the source list and the target list. @@ -518,8 +496,6 @@ To select and move multiple cards:  ---- - ### Issue ordering in a list When visiting a board, issues appear ordered in any list. You're able to change diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index 6cc31a45309..c721bef8f4d 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -31,7 +31,8 @@ git commit -m "this is my commit message. Related to https://gitlab.com/<usernam Of course, you can replace `gitlab.com` with the URL of your own GitLab instance. -NOTE: **Note:** Linking your first commit to your issue is going to be relevant +NOTE: **Note:** +Linking your first commit to your issue is going to be relevant for tracking your process with [GitLab Cycle Analytics](https://about.gitlab.com/stages-devops-lifecycle/value-stream-analytics/). It will measure the time taken for planning the implementation of that issue, which is the time between creating an issue and making the first commit. diff --git a/doc/user/project/issues/csv_import.md b/doc/user/project/issues/csv_import.md index d67b135186f..807f4fcffdf 100644 --- a/doc/user/project/issues/csv_import.md +++ b/doc/user/project/issues/csv_import.md @@ -7,7 +7,8 @@ Issues can be imported to a project by uploading a CSV file with the columns The user uploading the CSV file will be set as the author of the imported issues. -NOTE: **Note:** A permission level of [Developer](../../permissions.md), or higher, is required +NOTE: **Note:** +A permission level of [Developer](../../permissions.md), or higher, is required to import issues. ## Prepare for the import diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index ac397592a3b..618506ea7ee 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -30,9 +30,11 @@ to be enabled: project level, navigate to your project's **Settings > General**, expand **Visibility, project features, permissions** and enable **Git Large File Storage**. -Design Management requires that projects are using -[hashed storage](../../../administration/repository_storage_types.md#hashed-storage) -(the default storage type since v10.0). +Design Management also requires that projects are using +[hashed storage](../../../administration/raketasks/storage.md#migrate-to-hashed-storage). Since + GitLab 10.0, newly created projects use hashed storage by default. A GitLab admin can verify the storage type of a +project by navigating to **Admin Area > Projects** and then selecting the project in question. +A project can be identified as hashed-stored if its *Gitaly relative path* contains `@hashed`. If the requirements are not met, the **Designs** tab displays a message to the user. @@ -47,6 +49,7 @@ and [PDFs](https://gitlab.com/gitlab-org/gitlab/-/issues/32811) is planned for a ## Limitations - Design uploads are limited to 10 files at a time. +- From GitLab 13.1, Design filenames are limited to 255 characters. - Design Management data [isn't deleted when a project is destroyed](https://gitlab.com/gitlab-org/gitlab/-/issues/13429) yet. - Design Management data [won't be moved](https://gitlab.com/gitlab-org/gitlab/-/issues/13426) @@ -57,20 +60,62 @@ and [PDFs](https://gitlab.com/gitlab-org/gitlab/-/issues/32811) is planned for a - Only the latest version of the designs can be deleted. - Deleted designs cannot be recovered but you can see them on previous designs versions. -## The Design Management page +## GitLab-Figma plugin -Navigate to the **Design Management** page from any issue by clicking the **Designs** tab: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-figma-plugin/-/issues/2) in GitLab 13.2. - +Connect your design environment with your source code management in a seamless workflow. The GitLab-Figma plugin makes it quick and easy to collaborate in GitLab by bringing the work of product designers directly from Figma to GitLab Issues as uploaded Designs. + +To use the plugin, install it from the [Figma Directory](https://www.figma.com/community/plugin/860845891704482356) +and connect to GitLab through a personal access token. The details are explained in the [plugin documentation](https://gitlab.com/gitlab-org/gitlab-figma-plugin/-/wikis/home). + +## The Design Management section + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223193) in GitLab 13.2, Designs are displayed directly on the issue description rather than on a separate tab. +> - The new display is deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It cannot be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-displaying-designs-on-the-issue-description-core-only). If disabled, it will move Designs back to the **Designs** tab. + +You can find to the **Design Management** section in the issue description: + + + +### Enable or disable displaying Designs on the issue description **(CORE ONLY)** + +Displaying Designs on the issue description is under development but ready for +production use. It is deployed behind a feature flag that is **enabled by +default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it for your instance. + +To disable it: + +```ruby +Feature.disable(:design_management_moved) +``` + +To enable it: + +```ruby +Feature.enable(:design_management_moved) +``` + +By disabling this feature, designs will be displayed on the **Designs** tab +instead of directly on the issue description. ## Adding designs -To upload design images, click the **Upload Designs** button and select images to upload. +To upload Design images, drag files from your computer and drop them in the Design Management section, +or click **upload** to select images from your file browser: + + [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34353) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.9, you can drag and drop designs onto the dedicated drop zone to upload them. - + [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202634) in GitLab 12.10, you can also copy images from your file system and @@ -239,3 +284,13 @@ To disable it: ```ruby Feature.disable(:design_management_reference_filter_gfm_pipeline) ``` + +## Design activity records + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33051) in GitLab 13.1. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/225205) in GitLab 13.2. + +User activity events on designs (creation, deletion, and updates) are tracked by GitLab and +displayed on the [user profile](../../profile/index.md#user-profile), +[group](../../group/index.md#view-group-activity), +and [project](../index.md#project-activity) activity pages. diff --git a/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png b/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png Binary files differnew file mode 100644 index 00000000000..d60f1234b6d --- /dev/null +++ b/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png diff --git a/doc/user/project/issues/img/design_management_upload_v13.2.png b/doc/user/project/issues/img/design_management_upload_v13.2.png Binary files differnew file mode 100644 index 00000000000..1d4b10307fc --- /dev/null +++ b/doc/user/project/issues/img/design_management_upload_v13.2.png diff --git a/doc/user/project/issues/img/design_management_v13_2.png b/doc/user/project/issues/img/design_management_v13_2.png Binary files differnew file mode 100644 index 00000000000..0a6e2be17ab --- /dev/null +++ b/doc/user/project/issues/img/design_management_v13_2.png diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 06a80672769..9113f5344ba 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -175,8 +175,6 @@ requires [GraphQL](../../../api/graphql/index.md) to be enabled.  ---- - ### Health status **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36427) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 6d34f91d37f..7f236d04fb6 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -88,7 +88,7 @@ An issue can be assigned to: - Yourself. - Another person. -- [Many people](#multiple-assignees-STARTER). **(STARTER)** +- [Many people](#multiple-assignees-starter). **(STARTER)** The assignee(s) can be changed as often as needed. The idea is that the assignees are responsible for that issue until it's reassigned to someone else to take it from there. @@ -253,7 +253,7 @@ Also: ### Create Merge Request -Create a new branch and [WIP merge request](../merge_requests/work_in_progress_merge_requests.md) +Create a new branch and [**Draft** merge request](../merge_requests/work_in_progress_merge_requests.md) in one action. The branch will be named `issuenumber-title` by default, but you can choose any name, and GitLab will verify that it is not already in use. The merge request will automatically inherit the milestone and labels of the issue, and will be set to diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 08e3164b2eb..babc5030337 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -45,8 +45,7 @@ There are many ways to get to the New Issue form from within a project: ### Elements of the New Issue form -> Ability to add the new issue to an epic [was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) -> in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. +> Ability to add the new issue to an epic [was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1.  @@ -76,7 +75,7 @@ create issues for the same project.  -### New issue via Service Desk **(STARTER)** +### New issue via Service Desk Enable [Service Desk](../service_desk.md) for your project and offer email support. By doing so, when your customer sends a new email, a new issue can be created in diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 406938519b1..9886ef91f16 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -54,7 +54,7 @@ and edit labels. View the project labels list by going to the project and clicking **Issues > Labels**. The list includes all labels that are defined at the project level, as well as all -labels inherited from the parent group. You can filter the list by entering a search +labels inherited from the immediate parent group. You can filter the list by entering a search query at the top and clicking search (**{search}**). To create a new project label: @@ -94,7 +94,7 @@ also be merged. All issues, merge requests, issue board lists, issue board filters, and label subscriptions with the old labels will be assigned to the new group label. -WARNING: **Caution:** +CAUTION: **Caution:** Promoting a label is a permanent action, and cannot be reversed. To promote a project label to a group label: @@ -251,3 +251,16 @@ If you sort by `Priority`, GitLab uses this sort comparison order: Ties are broken arbitrarily.  + +## Troubleshooting + +### Some label titles end with `_duplicate<number>` + +In specific circumstances it was possible to create labels with duplicate titles in the same +namespace. + +To resolve the duplication, [in GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21384) +and later, some duplicate labels have `_duplicate<number>` appended to their titles. + +You can safely change these labels' titles if you prefer. +For details of the original problem, see [issue 30390](https://gitlab.com/gitlab-org/gitlab/issues/30390). diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 787a74b0065..3eb411aef18 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -128,3 +128,30 @@ If you change your mind before your request is approved, just click the ## Share project with group Alternatively, you can [share a project with an entire group](share_project_with_groups.md) instead of adding users one by one. + +## Remove a member from the project + +Only users with permissions of [Owner](../../permissions.md#group-members-permissions) can manage +project members. + +You can remove a user from the project if the given member has a direct membership in the project. +If membership is inherited from a parent group, then the member can be removed only from the parent +group itself. + +When removing a member, you can decide whether to unassign the user from all issues and merge +requests they are currently assigned or leave the assignments as they are. + +- **Unassigning the removed member** from all issues and merge requests might be helpful when a user + is leaving a private project and you wish to revoke their access to any issues and merge requests + they are assigned. +- **Keeping the issues and merge requests assigned** might be helpful for projects that accept public + contributions where a user doesn't have to be a member to be able to contribute to issues and + merge requests. + +To remove a member from a project: + +1. In a project, go to **{users}** **Members**. +1. Click the **Delete** **{remove}** button next to a project member you want to remove. + A **Remove member** modal appears. +1. (Optional) Select the **Also unassign this user from related issues and merge requests** checkbox. +1. Click **Remove member**. diff --git a/doc/user/project/merge_requests/accessibility_testing.md b/doc/user/project/merge_requests/accessibility_testing.md index 417b85a6082..f3a0aac9ff4 100644 --- a/doc/user/project/merge_requests/accessibility_testing.md +++ b/doc/user/project/merge_requests/accessibility_testing.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- @@ -20,7 +23,10 @@ analyzed to a file called `accessibility`. ## Accessibility Merge Request widget -[Since GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/39425), in addition to the report artifact that is created, GitLab will also show the +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39425) in GitLab 13.0 behind the disabled [feature flag](../../../administration/feature_flags.md) `:accessibility_report_view`. +> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217372) in GitLab 13.1. + +In addition to the report artifact that is created, GitLab will also show the Accessibility Report in the merge request widget area:  diff --git a/doc/user/project/merge_requests/browser_performance_testing.md b/doc/user/project/merge_requests/browser_performance_testing.md index 390d480724d..10457e40e0b 100644 --- a/doc/user/project/merge_requests/browser_performance_testing.md +++ b/doc/user/project/merge_requests/browser_performance_testing.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- @@ -7,20 +10,16 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. If your application offers a web interface and you're using -[GitLab CI/CD](../../../ci/README.md), you can quickly determine the performance -impact of pending code changes. +[GitLab CI/CD](../../../ci/README.md), you can quickly determine the rendering performance +impact of pending code changes in the browser. ## Overview GitLab uses [Sitespeed.io](https://www.sitespeed.io), a free and open source -tool, for measuring the performance of web sites. GitLab has built a simple -[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) which outputs -the performance score for each page analyzed in a file called `performance.json`. -The [Sitespeed.io performance score](https://examples.sitespeed.io/6.0/2017-11-23-23-43-35/help.html) -is a composite value based on best practices. - -GitLab can [show the Performance report](#how-browser-performance-testing-works) -in the merge request widget area. +tool, for measuring the rendering performance of web sites. The +[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) that GitLab built outputs +the performance score for each page analyzed in a file called `browser-performance.json` +this data can be shown on Merge Requests. ## Use cases @@ -38,7 +37,7 @@ Consider the following workflow: ## How browser performance testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). +[Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). GitLab then checks this report, compares key performance metrics for each page between the source and target branches, and shows the information in the merge request. @@ -46,12 +45,13 @@ For an example Performance job, see [Configuring Browser Performance Testing](#configuring-browser-performance-testing). NOTE: **Note:** -If the Performance report has no data to compare, such as when you add the -Performance job in your `.gitlab-ci.yml` for the very first time, no information -displays in the merge request widget area. Consecutive merge requests will have data for -comparison, and the Performance report will be shown properly. +If the Browser Performance report has no data to compare, such as when you add the +Browser Performance job in your `.gitlab-ci.yml` for the very first time, +the Browser Performance report widget won't show. It must have run at least +once on the target branch (`master`, for example), before it will display in a +merge request targeting that branch. - + ## Configuring Browser Performance Testing @@ -61,21 +61,7 @@ using Docker-in-Docker. 1. First, set up GitLab Runner with a [Docker-in-Docker build](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). -1. After configuring the Runner, add a new job to `.gitlab-ci.yml` that generates - the expected report. -1. Define the `performance` job according to your version of GitLab: - - - For GitLab 12.4 and later - [include](../../../ci/yaml/README.md#includetemplate) the - [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml) provided as a part of your GitLab installation. - - For GitLab versions earlier than 12.4 - Copy and use the job as defined in the - [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). - - CAUTION: **Caution:** - The job definition provided by the template does not support Kubernetes yet. - For a complete example of a more complex setup that works in Kubernetes, see - [`Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml). - -1. Add the following to your `.gitlab-ci.yml` file: +1. Configure the default Browser Performance Testing CI job as follows in your `.gitlab-ci.yml` file: ```yaml include: @@ -86,24 +72,32 @@ using Docker-in-Docker. URL: https://example.com ``` - CAUTION: **Caution:** - The job definition provided by the template is supported in GitLab 11.5 and later versions. - It also requires GitLab Runner 11.5 or later. For earlier versions, use the - [previous job definitions](#previous-job-definitions). +NOTE: **Note:** +For versions before 12.4, see the information for [older GitLab versions](#gitlab-versions-123-and-older). +If you are using a Kubernetes cluster, use [`template: Jobs/Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml) +instead. The above example creates a `performance` job in your CI/CD pipeline and runs sitespeed.io against the webpage you defined in `URL` to gather key metrics. -The [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance) -is downloaded to save the report as a [Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) -that you can later download and analyze. Due to implementation limitations, we always -take the latest Performance artifact available. -The full HTML sitespeed.io report is saved as an artifact, and if -[GitLab Pages](../pages/index.md) is enabled, it can be viewed directly in your browser. +The example uses a CI/CD template that is included in all GitLab installations since +12.4, but it will not work with Kubernetes clusters. If you are using GitLab 12.3 +or older, you must [add the configuration manually](#gitlab-versions-123-and-older) + +The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance), +and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) +that you can later download and analyze. This implementation always takes the latest +Browser Performance artifact available. If [GitLab Pages](../pages/index.md) is enabled, +you can view the report directly in your browser. + +You can also customize the jobs with environment variables: + +- `SITESPEED_IMAGE`: Configure the Docker image to use for the job (default `sitespeedio/sitespeed.io`), but not the image version. +- `SITESPEED_VERSION`: Configure the version of the Docker image to use for the job (default `13.3.0`). +- `SITESPEED_OPTIONS`: Configure any additional sitespeed.io options as required (default `nil`). Refer to the [sitespeed.io documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) for more details. -You can also customize options by setting the `SITESPEED_OPTIONS` variable. For example, you can override the number of runs sitespeed.io -makes on the given URL: +makes on the given URL, and change the version: ```yaml include: @@ -111,18 +105,11 @@ include: performance: variables: - URL: https://example.com + URL: https://www.sitespeed.io/ + SITESPEED_VERSION: 13.2.0 SITESPEED_OPTIONS: -n 5 ``` -For further customization options for sitespeed.io, including the ability to provide a -list of URLs to test, please see the -[Sitespeed.io Configuration](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) -documentation. - -TIP: **Tip:** -Key metrics are automatically extracted and shown in the merge request widget. - ### Configuring degradation threshold > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27599) in GitLab 13.0. @@ -149,15 +136,12 @@ The above CI YAML configuration is great for testing against static environments be extended for dynamic environments, but a few extra steps are required: 1. The `performance` job should run after the dynamic environment has started. -1. In the `review` job, persist the hostname and upload it as an artifact so - it's available to the `performance` job. The same can be done for static - environments like staging and production to unify the code path. You can save it - as an artifact with `echo $CI_ENVIRONMENT_URL > environment_url.txt` - in your job's `script`. -1. In the `performance` job, read the previous artifact into an environment - variable. In this case, use `$URL` because the sitespeed.io command - uses it for the URL parameter. Because Review App URLs are dynamic, define - the `URL` variable through `before_script` instead of `variables`. +1. In the `review` job: + 1. Generate a URL list file with the dynamic URL. + 1. Save the file as an artifact, for example with `echo $CI_ENVIRONMENT_URL > environment_url.txt` + in your job's `script`. + 1. Pass the list as the URL environment variable (which can be a URL or a file containing URLs) + to the `performance` job. 1. You can now run the sitespeed.io container against the desired hostname and paths. @@ -190,20 +174,21 @@ review: performance: dependencies: - review - before_script: - - export URL=$(cat environment_url.txt) + variables: + URL: environment_url.txt ``` -### Previous job definitions +### GitLab versions 12.3 and older -CAUTION: **Caution:** -Before GitLab 11.5, the Performance job and artifact had to be named specifically -to automatically extract report data and show it in the merge request widget. -While these old job definitions are still maintained, they have been deprecated -and may be removed in next major release, GitLab 12.0. -GitLab recommends you update your current `.gitlab-ci.yml` configuration to reflect that change. +Browser Performance Testing has gone through several changes since it's introduction. +In this section we'll detail these changes and how you can run the test based on your +GitLab version: -For GitLab 11.4 and earlier, the job should look like: +- In GitLab 12.4 [a job template was made available](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). +- In 13.2 the feature was renamed from `Performance` to `Browser Performance` with +additional template variables. The job name in the template is still `performance` +for compatibility reasons, but may be renamed to match in a future iteration. +- For 11.5 to 12.3 no template is available and the job has to be defined manually as follows: ```yaml performance: @@ -211,28 +196,45 @@ performance: image: docker:git variables: URL: https://example.com + SITESPEED_VERSION: 13.3.0 + SITESPEED_OPTIONS: '' services: - docker:stable-dind script: - mkdir gitlab-exporter - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js - mkdir sitespeed-results - - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:$SITESPEED_VERSION --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL $SITESPEED_OPTIONS - mv sitespeed-results/data/performance.json performance.json artifacts: paths: - performance.json - sitespeed-results/ + reports: + performance: performance.json ``` -<!-- ## Troubleshooting +- For 11.4 and earlier the job should be defined as follows: -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +```yaml +performance: + stage: performance + image: docker:git + variables: + URL: https://example.com + services: + - docker:stable-dind + script: + - mkdir gitlab-exporter + - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js + - mkdir sitespeed-results + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL + - mv sitespeed-results/data/performance.json performance.json + artifacts: + paths: + - performance.json + - sitespeed-results/ +``` -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +Upgrading to the latest version and using the templates is recommended, to ensure +you receive the latest updates, including updates to the sitespeed.io versions. diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 7b4d4b668d5..36acba032ff 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -1,8 +1,11 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- -# Code Quality **(STARTER)** +# Code Quality > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. @@ -22,8 +25,13 @@ Code Quality: DevOps](../../../topics/autodevops/stages.md#auto-code-quality-starter). - Can be extended through [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) or a [custom tool](#implementing-a-custom-tool). +## Code Quality Widget + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in 13.2. + Going a step further, GitLab can show the Code Quality report right -in the merge request widget area: +in the merge request widget area if a report from the target branch is available to compare to:  @@ -79,7 +87,7 @@ include: The above example will create a `code_quality` job in your CI/CD pipeline which will scan your source code for code quality issues. The report will be saved as a -[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter) +[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality) that you can later download and analyze. It's also possible to override the URL to the Code Quality image by @@ -237,7 +245,7 @@ do this: 1. Define a job in your `.gitlab-ci.yml` file that generates the [Code Quality report - artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter). + artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality). 1. Configure your tool to generate the Code Quality report artifact as a JSON file that implements a subset of the [Code Climate spec](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types). @@ -273,11 +281,11 @@ NOTE: **Note:** Although the Code Climate spec supports more properties, those are ignored by GitLab. -## Code Quality reports +## Code Quality reports **(STARTER)** Once the Code Quality job has completed: -- The full list of code quality violations generated by a pipeline is available in the +- The full list of code quality violations generated by a pipeline is shown in the Code Quality tab of the Pipeline Details page. - Potential changes to code quality are shown directly in the merge request. The Code Quality widget in the merge request compares the reports from the base and head of the branch, @@ -286,8 +294,43 @@ Once the Code Quality job has completed: [downloadable artifact](../../../ci/pipelines/job_artifacts.md#downloading-artifacts) for the `code_quality` job. +## Extending functionality + +### Using Analysis Plugins + +Should there be a need to extend the default functionality provided by Code Quality, as stated in [Code Quality](#code-quality), [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) are available. + +For example, to use the [SonarJava analyzer](https://docs.codeclimate.com/docs/sonar-java), +add a file named `.codeclimate.yml` containing the [enablement code](https://docs.codeclimate.com/docs/sonar-java#enable-the-plugin) +for the plugin to the root of your repository: + +```yaml +version: "2" +plugins: + sonar-java: + enabled: true +``` + +This adds SonarJava to the `plugins:` section of the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml) +included in your project. + +Changes to the `plugins:` section do not affect the `exclude_patterns` section of the +defeault `.codeclimate.yml`. See the Code Climate documentation for +[excluding files and folders](https://docs.codeclimate.com/docs/excluding-files-and-folders) +for more details. + +Here's [an example project](https://gitlab.com/jheimbuck_gl/jh_java_example_project) that uses Code Quality with a `.codeclimate.yml` file. + ## Troubleshooting +### Changing the default configuration has no effect + +A common issue is that the terms `Code Quality` (GitLab specific) and `Code Climate` +(Engine used by GitLab) are very similar. You must add a **`.codeclimate.yml`** file +to change the default configuration, **not** a `.codequality.yml` file. If you use +the wrong filename, the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml) +is still used. + ### No Code Quality report is displayed in a Merge Request This can be due to multiple reasons: @@ -295,6 +338,7 @@ This can be due to multiple reasons: - You just added the Code Quality job in your `.gitlab-ci.yml`. The report does not have anything to compare to yet, so no information can be displayed. Future merge requests will have something to compare to. +- Your pipeline is not set to run the code quality job on your default branch. If there is no report generated from the default branch, your MR branch reports will not have anything to compare to. - If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks), nothing will be displayed. - The [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) CI/CD diff --git a/doc/user/project/merge_requests/fail_fast_testing.md b/doc/user/project/merge_requests/fail_fast_testing.md new file mode 100644 index 00000000000..619a6d04577 --- /dev/null +++ b/doc/user/project/merge_requests/fail_fast_testing.md @@ -0,0 +1,87 @@ +--- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Fail Fast Testing **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198550) in GitLab 13.1. + +For applications that use RSpec for running tests, we've introduced the `Verify/Failfast` +[template to run subsets of your test suite](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Verify/FailFast.gitlab-ci.yml), +based on the changes in your merge request. + +The template uses the [test_file_finder (`tff`) gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder/) +that accepts a list of files as input, and returns a list of spec (test) files +that it believes to be relevant to the input files. + +`tff` is designed for Ruby on Rails projects, so the `Verify/FailFast` template is +configured to run when changes to Ruby files are detected. By default, it runs in +the [`.pre` stage](../../../ci/yaml/README.md#pre-and-post) of a GitLab CI/CD pipeline, +before all other stages. + +## Example use case + +Fail fast testing is useful when adding new functionality to a project and adding +new automated tests. + +Your project could have hundreds of thousands of tests that take a long time to complete. +You may be confident that a new test will pass, but you have to wait for all the tests +to complete to verify it. This could take an hour or more, even when using parallelization. + +Fail fast testing gives you a faster feedback loop from the pipeline. It lets you +know quickly that the new tests are passing and the new functionality did not break +other tests. + +## Requirements + +This template requires: + +- A project built in Rails that uses RSpec for testing. +- CI/CD configured to: + - Use a Docker image with Ruby available. + - Use [Pipelines for Merge Requests](../../../ci/merge_request_pipelines/index.md#configuring-pipelines-for-merge-requests) +- [Pipelines for Merged Results](../../../ci/merge_request_pipelines/pipelines_for_merged_results/index.md#enable-pipelines-for-merged-results) + enabled in the project settings. + +## Configure Fast RSpec Failure + +We'll use the following plain RSpec configuration as a starting point. It installs all the +project gems and executes `rspec`, on merge request pipelines only. + +```yaml +rspec-complete: + stage: test + rules: + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + script: + - bundle install + - bundle exec rspec +``` + +To run the most relevant specs first instead of the whole suite, [`include`](../../../ci/yaml/README.md#include) +the template by adding the following to your CI/CD configuration: + +```yaml +include: + - template: Verify/FailFast.gitlab-ci.yml +``` + +### Example test loads + +For illustrative purposes, let's say our Rails app spec suite consists of 100 specs per model for ten models. + +If no Ruby files are changed: + +- `rspec-rails-modified-paths-specs` will not run any tests. +- `rspec-complete` will run the full suite of 1000 tests. + +If one Ruby model is changed, for example `app/models/example.rb`, then `rspec-rails-modified-paths-specs` +will run the 100 tests for `example.rb`: + +- If all of these 100 tests pass, then the full `rspec-complete` suite of 1000 tests is allowed to run. +- If any of these 100 tests fail, they will fail quickly, and `rspec-complete` will not run any tests. + +The final case saves resources and time as the full 1000 test suite does not run. diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index 32eb0c73ed4..9ac0f3ee42e 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -57,7 +57,7 @@ request's page at the top-right side: - [Close issues automatically](#merge-requests-to-close-issues) when they are merged. - Enable the [delete source branch when merge request is accepted](#deleting-the-source-branch) option to keep your repository clean. - Enable the [squash commits when merge request is accepted](squash_and_merge.md) option to combine all the commits into one before merging, thus keep a clean commit history in your repository. -- Set the merge request as a [Work In Progress (WIP)](work_in_progress_merge_requests.md) to avoid accidental merges before it is ready. +- Set the merge request as a [**Draft**](work_in_progress_merge_requests.md) to avoid accidental merges before it is ready. Once you have created the merge request, you can also: diff --git a/doc/user/project/merge_requests/img/browser_performance_testing.png b/doc/user/project/merge_requests/img/browser_performance_testing.png Binary files differindex eea77fb8b93..c270462f7a8 100644 --- a/doc/user/project/merge_requests/img/browser_performance_testing.png +++ b/doc/user/project/merge_requests/img/browser_performance_testing.png diff --git a/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png b/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png Binary files differnew file mode 100644 index 00000000000..beb12c581d6 --- /dev/null +++ b/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png diff --git a/doc/user/project/merge_requests/img/file_by_file_v13_2.png b/doc/user/project/merge_requests/img/file_by_file_v13_2.png Binary files differnew file mode 100644 index 00000000000..e3114ebabad --- /dev/null +++ b/doc/user/project/merge_requests/img/file_by_file_v13_2.png diff --git a/doc/user/project/merge_requests/img/load_performance_testing.png b/doc/user/project/merge_requests/img/load_performance_testing.png Binary files differnew file mode 100644 index 00000000000..3a58e9c28d4 --- /dev/null +++ b/doc/user/project/merge_requests/img/load_performance_testing.png diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png Binary files differdeleted file mode 100644 index 761690d1e0c..00000000000 --- a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png Binary files differdeleted file mode 100644 index ab2c8425b83..00000000000 --- a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png +++ /dev/null diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 5d2813f5bfc..f68fc7d7b45 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -19,7 +19,7 @@ A. Consider you're a software developer working in a team: 1. You checkout a new branch, and submit your changes through a merge request 1. You gather feedback from your team -1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) **(STARTER)** +1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) 1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD 1. You avoid using dependencies whose license is not compatible with your project with [License Compliance reports](../../compliance/license_compliance/index.md) **(ULTIMATE)** 1. You request the [approval](merge_request_approvals.md) from your manager **(STARTER)** diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md new file mode 100644 index 00000000000..3239269109d --- /dev/null +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -0,0 +1,197 @@ +--- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Load Performance Testing **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +With Load Performance Testing, you can test the impact of any pending code changes +to your application's backend in [GitLab CI/CD](../../../ci/README.md). + +GitLab uses [k6](https://k6.io/), a free and open source +tool, for measuring the system performance of applications under +load. + +Unlike [Browser Performance Testing](browser_performance_testing.md), which is +used to measure how web sites perform in client browsers, Load Performance Testing +can be used to perform various types of [load tests](https://k6.io/docs/#use-cases) +against application endpoints such as APIs, Web Controllers, and so on. +This can be used to test how the backend or the server performs at scale. + +For example, you can use Load Performance Testing to perform many concurrent +GET calls to a popular API endpoint in your application to see how it performs. + +## How Load Performance Testing works + +First, define a job in your `.gitlab-ci.yml` file that generates the +[Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +GitLab checks this report, compares key load performance metrics +between the source and target branches, and then shows the information in a merge request widget: + + + +Next, you need to configure the test environment and write the k6 test. + +The key performance metrics that the merge request widget shows after the test completes are: + +- Checks: The percentage pass rate of the [checks](https://k6.io/docs/using-k6/checks) configured in the k6 test. +- TTFB P90: The 90th percentile of how long it took to start receiving responses, aka the [Time to First Byte](https://en.wikipedia.org/wiki/Time_to_first_byte) (TTFB). +- TTFB P95: The 95th percentile for TTFB. +- RPS: The average requests per second (RPS) rate the test was able to achieve. + +NOTE: **Note:** +If the Load Performance report has no data to compare, such as when you add the +Load Performance job in your `.gitlab-ci.yml` for the very first time, +the Load Performance report widget won't show. It must have run at least +once on the target branch (`master`, for example), before it will display in a +merge request targeting that branch. + +## Configure the Load Performance Testing job + +Configuring your Load Performance Testing job can be broken down into several distinct parts: + +- Determine the test parameters such as throughput, and so on. +- Set up the target test environment for load performance testing. +- Design and write the k6 test. + +### Determine the test parameters + +The first thing you need to do is determine the [type of load test](https://k6.io/docs/test-types/introduction) +you want to run, and how it will run (for example, the number of users, throughput, and so on). + +Refer to the [k6 docs](https://k6.io/docs/), especially the [k6 testing guides](https://k6.io/docs/testing-guides), +for guidance on the above and more. + +### Test Environment setup + +A large part of the effort around load performance testing is to prepare the target test environment +for high loads. You should ensure it's able to handle the +[throughput](https://k6.io/blog/monthly-visits-concurrent-users) it will be tested with. + +It's also typically required to have representative test data in the target environment +for the load performance test to use. + +We strongly recommend [not running these tests against a production environment](https://k6.io/our-beliefs#load-test-in-a-pre-production-environment). + +### Write the load performance test + +After the environment is prepared, you can write the k6 test itself. k6 is a flexible +tool and can be used to run [many kinds of performance tests](https://k6.io/docs/test-types/introduction). +Refer to the [k6 documentation](https://k6.io/docs/) for detailed information on how to write tests. + +### Configure the test in GitLab CI/CD + +When your k6 test is ready, the next step is to configure the load performance +testing job in GitLab CI/CD. The easiest way to do this is to use the +[`Verify/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Load-Performance-Testing.gitlab-ci.yml) +template that is included with GitLab. + +NOTE: **Note:** +For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual +test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) +for spec details. The [default shared GitLab.com runners](../../gitlab_com/#linux-shared-runners) +likely have insufficient specs to handle most large k6 tests. + +This template runs the +[k6 Docker container](https://hub.docker.com/r/loadimpact/k6/) in the job and provides several ways to customize the +job. + +An example configuration workflow: + +1. Set up a GitLab Runner that can run Docker containers, such as a Runner using the + [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). +1. Configure the default Load Performance Testing CI job in your `.gitlab-ci.yml` file. + You need to include the template and configure it with variables: + + ```yaml + include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + + load_performance: + variables: + K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT> + ``` + +The above example creates a `load_performance` job in your CI/CD pipeline that runs +the k6 test. + +NOTE: **Note:** +For Kubernetes setups a different template should be used: [`Jobs/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Load-Performance-Testing.gitlab-ci.yml). + +k6 has [various options](https://k6.io/docs/using-k6/options) to configure how it will run tests, such as what throughput (RPS) to run with, +how long the test should run, and so on. Almost all options can be configured in the test itself, but as +you can also pass command line options via the `K6_OPTIONS` variable. + +For example, you can override the duration of the test with a CLI option: + +```yaml + include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + + load_performance: + variables: + K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT> + K6_OPTIONS: '--duration 30s' +``` + +GitLab only displays the key performance metrics in the MR widget if k6's results are saved +via [summary export](https://k6.io/docs/results-visualization/json#summary-export) +as a [Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +The latest Load Performance artifact available is always used. + +If [GitLab Pages](../pages/index.md) is enabled, you can view the report directly in your browser. + +### Load Performance testing in Review Apps + +The CI/CD YAML configuration example above works for testing against static environments, +but it can be extended to work with [review apps](../../../ci/review_apps) or +[dynamic environments](../../../ci/environments) with a few extra steps. + +The best approach is to capture the dynamic URL into a custom environment variable that +is then [inherited](../../../ci/variables/README.md#inherit-environment-variables) +by the `load_performance` job. The k6 test script to be run should then be configured to +use that environment URL, such as: ``http.get(`${__ENV.ENVIRONMENT_URL`})``. + +For example: + +1. In the `review` job: + 1. Capture the dynamic URL and save it into a `.env` file, e.g. `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. + 1. Set the `.env` file to be an [`artifacts:reports:dotenv` report](../../../ci/variables/README.md#inherit-environment-variables). +1. Set the `load_performance` job to depend on the review job, so it inherits the environment variable. +1. Configure the k6 test script to use the environment variable in it's steps. + +Your `.gitlab-ci.yml` file might be similar to: + +```yaml +stages: + - deploy + - performance + +include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + +review: + stage: deploy + environment: + name: review/$CI_COMMIT_REF_NAME + url: http://$CI_ENVIRONMENT_SLUG.example.com + script: + - run_deploy_script + - echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env + artifacts: + reports: + dotenv: + review.env + rules: + - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + +load_performance: + dependencies: + - review + rules: + - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. +``` diff --git a/doc/user/project/merge_requests/merge_request_dependencies.md b/doc/user/project/merge_requests/merge_request_dependencies.md index a1012e27d32..65f3cb1e0d5 100644 --- a/doc/user/project/merge_requests/merge_request_dependencies.md +++ b/doc/user/project/merge_requests/merge_request_dependencies.md @@ -38,15 +38,15 @@ the `awesome-project` merge request before the `awesome-lib` one would break the `master`branch. The `awesome-project` merge request could be [marked as -WIP](work_in_progress_merge_requests.md), -and the reason for the WIP stated included in the comments. However, this +**Draft**](work_in_progress_merge_requests.md), +and the reason for the draft stated included in the comments. However, this requires the state of the `awesome-lib` merge request to be manually tracked, and doesn't scale well if the `awesome-project` merge request depends on changes to **several** other projects. By making the `awesome-project` merge request depend on the `awesome-lib` merge request instead, this relationship is -automatically tracked by GitLab, and the WIP state can be used to +automatically tracked by GitLab, and the draft state can be used to communicate the readiness of the code in each individual merge request instead. diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index d45ccdc9be9..7d90c9f3cd7 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -1,36 +1,38 @@ --- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, concepts --- # Merge when pipeline succeeds -When reviewing a merge request that looks ready to merge but still has one or -more CI jobs running, you can set it to be merged automatically when the -jobs pipeline succeeds. This way, you don't have to wait for the jobs to +When reviewing a merge request that looks ready to merge but still has a +pipeline running, you can set it to merge automatically when the +pipeline succeeds. This way, you don't have to wait for the pipeline to finish and remember to merge the request manually.  ## How it works -When you hit the "Merge When Pipeline Succeeds" button, the status of the merge -request will be updated to represent the impending merge. If you cannot wait -for the pipeline to succeed and want to merge immediately, this option is -available in the dropdown menu on the right of the main button. +When you click "Merge When Pipeline Succeeds", the status of the merge +request is updated to show the impending merge. If you can't wait +for the pipeline to succeed, you can choose **Merge immediately** +in the dropdown menu on the right of the main button. -Both team developers and the author of the merge request have the option to -cancel the automatic merge if they find a reason why it shouldn't be merged -after all. +The author of the merge request and project members with developer permissions can +cancel the automatic merge at any time before the pipeline finishes.  -When the pipeline succeeds, the merge request will automatically be merged. +When the pipeline succeeds, the merge request is automatically merged. When the pipeline fails, the author gets a chance to retry any failed jobs, or to push new commits to fix the failure. When the jobs are retried and succeed on the second try, the merge request -will automatically be merged after all. When the merge request is updated with -new commits, the automatic merge is automatically canceled to allow the new +is automatically merged. When the merge request is updated with +new commits, the automatic merge is canceled to allow the new changes to be reviewed. ## Only allow merge requests to be merged if the pipeline succeeds @@ -42,7 +44,7 @@ or if there are threads to be resolved. This works for both: - Pipelines run from an [external CI integration](../integrations/overview.md#integrations-listing) As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md) -will not disable this feature, as it will still be possible to use pipelines from external +does not disable this feature, as it is possible to use pipelines from external CI providers with this feature. To enable it, you must: 1. Navigate to your project's **Settings > General** page. @@ -50,14 +52,40 @@ CI providers with this feature. To enable it, you must: 1. In the **Merge checks** subsection, select the **Pipelines must succeed** checkbox. 1. Press **Save** for the changes to take effect. -NOTE: **Note:** This setting also prevents merge requests from being merged if there is no pipeline. +This setting also prevents merge requests from being merged if there is no pipeline. - +### Limitations + +When this setting is enabled, a merge request is prevented from being merged if there +is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced) +or [`rules`](../../../ci/yaml/README.md#rules) are used and they don't generate any pipelines. + +You should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) +and that it's successful. + +If both a branch pipeline and a merge request pipeline are triggered for a single +merge request, only the success or failure of the *merge request pipeline* is checked. +If the merge request pipeline is configured with fewer jobs than the branch pipeline, +it could allow code that fails tests to be merged: + +```yaml +branch-pipeline-job: + rules: + - if: '$CI_PIPELINE_SOURCE == "push"' + script: + - echo "Code testing scripts here, for example." -From now on, every time the pipeline fails you will not be able to merge the -merge request from the UI, until you make all relevant jobs pass. +merge-request-pipeline-job: + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + script: + - echo "No tests run, but this pipeline always succeeds and enables merge." + - echo true +``` - +You should avoid configuration like this, and only use branch (`push`) pipelines +or merge request pipelines, when possible. See [`rules` documentation](../../../ci/yaml/README.md#differences-between-rules-and-onlyexcept) +for details on avoiding two pipelines for a single merge request. ### Skipped pipelines @@ -72,20 +100,10 @@ merge requests from being merged. To change this behavior: 1. In the **Merge checks** subsection, select the **Skipped pipelines are considered successful** checkbox. 1. Press **Save** for the changes to take effect. -### Limitations - -When this setting is enabled, a merge request is prevented from being merged if there is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced) rules are used and they don't generate any pipelines. - -Users that expect to be able to merge a merge request in this scenario should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) and that it's successful. +## From the command line -For example, to that on merge requests there is always a passing job even though `only/except` rules may not generate any other jobs: - -```yaml -enable_merge: - only: [merge_requests] - script: - - echo true -``` +You can use [Push Options](../push_options.md) to enable merge when pipeline succeeds +for a merge request when pushing from the command line. <!-- ## Troubleshooting @@ -98,8 +116,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -## Use it from the command line - -You can use [Push Options](../push_options.md) to trigger this feature when -pushing. diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index a3ad41d8dd8..162ebdf157d 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -64,6 +64,43 @@ list.  +### File-by-file diff navigation + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222790) in GitLab 13.2. +> - It's deployed behind a feature flag, disabled by default. +> - It's enabled on GitLab.com. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-file-by-file-diff-navigation-core-only). + +For larger merge requests it might sometimes be useful to review single files at a time. To enable, +from your avatar on the top-right navbar, click **Settings**, and go to **Preferences** on the left +sidebar. Scroll down to the **Behavior** section and select **Show one file at a time on merge request's Changes tab**. +Click **Save changes** to apply. + +From there, when reviewing merge requests' **Changes** tab, you will see only one file at a time. You can then click the buttons **Prev** and **Next** to view the other files changed. + + + +#### Enable or disable file-by-file diff navigation **(CORE ONLY)** + +File-by-file diff navigation is under development but ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:view_diffs_file_by_file) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:view_diffs_file_by_file>) +``` + ### Merge requests commit navigation > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18140) in GitLab 13.0. diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 924334055b9..79eec059293 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -85,6 +85,60 @@ it. This is because squashing is only available when accepting a merge request, so a merge request may need to be rebased before squashing, even though squashing can itself be considered equivalent to rebasing. +## Squash Commits Options + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17613) in GitLab 13.2. +> - It's deployed behind a feature flag, disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-squash-commit-options-core-only). **(CORE ONLY)** + +With Squash Commits Options you can configure the behavior of Squash and Merge for your project. +To set it up, navigate to your project's **Settings > General** and expand **Merge requests**. +You will find the following options to choose, which will affect existing and new merge requests +submitted to your project: + +- **Do not allow**: users cannot use Squash and Merge to squash all the commits immediately before + merging. The checkbox to enable or disable it will be unchecked and hidden from the users. +- **Allow**: users will have the option to enable Squash and Merge on a merge request basis. + The checkbox will be unchecked (disabled) by default, but and the user is allowed to enable it. +- **Encourage**: users will have the option to enable Squash and Merge on a merge request basis. + The checkbox will be checked (enabled) by default to encourage its use, but the user is allowed to + disable it. +- **Require**: Squash and Merge is enabled for all merge requests, so it will always be performed. + The checkbox to enable or disable it will be checked and hidden from the users. + +The Squash and Merge checkbox is displayed when you create a merge request and when you edit the description of an existing one, except when Squash Commit Options is set to **Do not allow** or **Require**. + +NOTE: **Note:** +If your project is set to **Do not allow** Squash and Merge, the users still have the option to +squash commits locally through the command line and force-push to their remote branch before merging. + +### Enable or disable Squash Commit Options **(CORE ONLY)** + +Squash Commit Options is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. Squash Commit Options can be enabled or disabled per-project. + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:squash_options) +# or by project +Feature.enable(:squash_options, Project.find(<project id>)) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:squash_options) +# or by project +Feature.disable(:squash_options, Project.find(<project id>)) +``` + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 1c0e698aba5..12194423a00 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- diff --git a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md index f7614ed7996..e5ebc46d58f 100644 --- a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md +++ b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: index description: "Test your code and display reports in merge requests" --- @@ -11,8 +14,9 @@ or link to useful information directly from merge requests: | Feature | Description | |--------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Accessibility Testing](accessibility_testing.md) | Automatically report A11y violations for changed pages in merge requests. | -| [Browser Performance Testing](browser_performance_testing.md) **(PREMIUM)** | Quickly determine the performance impact of pending code changes. | -| [Code Quality](code_quality.md) **(STARTER)** | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | +| [Browser Performance Testing](browser_performance_testing.md) **(PREMIUM)** | Quickly determine the browser performance impact of pending code changes. | +| [Load Performance Testing](load_performance_testing.md) **(PREMIUM)** | Quickly determine the server performance impact of pending code changes. | +| [Code Quality](code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | | [Display arbitrary job artifacts](../../../ci/yaml/README.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../../../ci/pipelines/job_artifacts.md) in merge requests. | | [GitLab CI/CD](../../../ci/README.md) | Build, test, and deploy your code in a per-branch basis with built-in CI/CD. | | [JUnit test reports](../../../ci/junit_test_reports.md) | Configure your CI jobs to use JUnit test reports, and let GitLab display a report on the merge request so that it’s easier and faster to identify the failure without having to check the entire job log. | diff --git a/doc/user/project/merge_requests/work_in_progress_merge_requests.md b/doc/user/project/merge_requests/work_in_progress_merge_requests.md index 8ac4131e10b..ece5e7868dc 100644 --- a/doc/user/project/merge_requests/work_in_progress_merge_requests.md +++ b/doc/user/project/merge_requests/work_in_progress_merge_requests.md @@ -2,42 +2,46 @@ type: reference, concepts --- -# "Work In Progress" merge requests +# Draft merge requests If a merge request is not yet ready to be merged, perhaps due to continued development or open threads, you can prevent it from being accepted before it's ready by flagging -it as a **Work In Progress**. This will disable the "Merge" button, preventing it from -being merged, and it will stay disabled until the "WIP" flag has been removed. +it as a **Draft**. This will disable the "Merge" button, preventing it from +being merged, and it will stay disabled until the "Draft" flag has been removed. - + -## Adding the "Work In Progress" flag to a merge request +## Adding the "Draft" flag to a merge request -There are several ways to flag a merge request as a Work In Progress: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32692) in GitLab 13.2, Work-In-Progress (WIP) merge requests were renamed to **Draft**. Support for using **WIP** will be removed in GitLab 14.0. -- Add `[WIP]` or `WIP:` to the start of the merge request's title. Clicking on - **Start the title with WIP:**, under the title box, when editing the merge request's +There are several ways to flag a merge request as a Draft: + +- Add `[Draft]`, `Draft:` or `(Draft)` to the start of the merge request's title. Clicking on + **Start the title with Draft:**, under the title box, when editing the merge request's description will have the same effect. +- **Deprecated** Add `[WIP]` or `WIP:` to the start of the merge request's title. + **WIP** still works but was deprecated in favor of **Draft**. It will be removed in the next major version (GitLab 14.0). - Add the `/wip` [quick action](../quick_actions.md#quick-actions-for-issues-merge-requests-and-epics) in a comment in the merge request. This is a toggle, and can be repeated to change the status back. Note that any other text in the comment will be discarded. -- Add "wip" or "WIP" to the start of a commit message targeting the merge request's +- Add `draft:` or `Draft:` to the start of a commit message targeting the merge request's source branch. This is not a toggle, and doing it again in another commit will have no effect. -## Removing the "Work In Progress" flag from a merge request +## Removing the "Draft" flag from a merge request Similar to above, when a Merge Request is ready to be merged, you can remove the -"Work in Progress" flag in several ways: +`Draft` flag in several ways: -- Remove `[WIP]` or `WIP:` from the start of the merge request's title. Clicking on - **Remove the WIP: prefix from the title**, under the title box, when editing the merge +- Remove `[Draft]`, `Draft:` or `(Draft)` from the start of the merge request's title. Clicking on + **Remove the Draft: prefix from the title**, under the title box, when editing the merge request's description, will have the same effect. - Add the `/wip` [quick action](../quick_actions.md#quick-actions-for-issues-merge-requests-and-epics) in a comment in the merge request. This is a toggle, and can be repeated to change the status back. Note that any other text in the comment will be discarded. -- Click on the **Resolve WIP status** button near the bottom of the merge request description, - next to the "Merge" button (see [image above](#work-in-progress-merge-requests)). +- Click on the **Resolve Draft status** button near the bottom of the merge request description, + next to the **Merge** button (see [image above](#draft-merge-requests)). Must have at least Developer level permissions on the project for the button to be visible. diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 421cb42de1e..aea5eef5efc 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -29,39 +29,67 @@ Similarly, milestones can be used as releases. To do so: 1. Set the milestone title to the version of your release, such as `Version 9.4`. 1. Add an issue to your release by associating the desired milestone from the issue's right-hand sidebar. -Additionally, you can integrate milestones with GitLab's [Releases feature](../releases/index.md#releases-associated-with-milestones). +Additionally, you can integrate milestones with GitLab's [Releases feature](../releases/index.md#associate-milestones-with-a-release). ## Project milestones and group milestones -- **Project milestones** can be assigned to issues or merge requests in that project only. Navigate to **Issues > Milestones** in a project to view the project milestone list. -- **Group milestones** can be assigned to any issue or merge request of any project in that group. Navigate to **Issues > Milestones** in a group to view the group milestone list. -- All milestones you have access to can also be viewed in the dashboard milestones list. Click on **Milestones** on the top navigation bar to view both project milestones and group milestones you have access to. +You can assign **project milestones** to issues or merge requests in that project only. +To view the project milestone list, in a project, go to **{issues}** **Issues > Milestones**. + +You can assign **group milestones** to any issue or merge request of any project in that group. +To view the group milestone list, in a group, go to **{issues}** **Issues > Milestones**. + +You can also view all milestones you have access to in the dashboard milestones list. +To view both project milestones and group milestones you have access to, click **More > Milestones** +on the top navigation bar. + +For information about project and group milestones API, see: + +- [Project Milestones API](../../../api/milestones.md) +- [Group Milestones API](../../../api/group_milestones.md) + +NOTE: **Note:** +If you're in a group and click **Issues > Milestones**, you'll see group milestones and the milestones +of projects in this group. +If you're in a project and click **Issues > Milestones**, you'll only see this project's milestones. ## Creating milestones ->**Note:** -A permission level of `Developer` or higher is required to create milestones. +NOTE: **Note:** +A permission level of [Developer or higher](../../permissions.md) is required to create milestones. ### New project milestone -To create a **project milestone**, navigate to **Issues > Milestones** in the project. +To create a **project milestone**: -Click the **New milestone** button. Enter the title, an optional description, an optional start date, and an optional due date. Click **Create milestone** to create the milestone. +1. In a project, go to **{issues}** **Issues > Milestones**. +1. Click **New milestone**. +1. Enter the title, an optional description, an optional start date, and an optional due date. +1. Click **New milestone**.  ### New group milestone -To create a **group milestone**, follow similar steps from above to project milestones. Navigate to **Issues > Milestones** in the group and create it from there. +To create a **group milestone**: + +1. In a group, go to **{issues}** **Issues > Milestones**. +1. Click **New milestone**. +1. Enter the title, an optional description, an optional start date, and an optional due date. +1. Click **New milestone**.  ## Editing milestones ->**Note:** -A permission level of `Developer` or higher is required to edit milestones. +NOTE: **Note:** +A permission level of [Developer or higher](../../permissions.md) is required to edit milestones. + +To edit a milestone: -You can update a milestone by navigating to **Issues > Milestones** in the project or group and clicking the **Edit** button. +1. In a project or group, go to **{issues}** **Issues > Milestones**. +1. Click a milestone's title. +1. Click **Edit**. You can delete a milestone by clicking the **Delete** button. diff --git a/doc/user/project/operations/alert_management.md b/doc/user/project/operations/alert_management.md index 411b36411af..3e4b94473bc 100644 --- a/doc/user/project/operations/alert_management.md +++ b/doc/user/project/operations/alert_management.md @@ -17,16 +17,64 @@ being developed, efficiency and awareness can be increased. NOTE: **Note:** You will need at least Maintainer [permissions](../../permissions.md) to enable the Alert Management feature. -1. Follow the [instructions for toggling generic alerts](../integrations/generic_alerts.md#setting-up-generic-alerts) -1. You can now visit **{cloud-gear}** **Operations > Alerts** in your project's sidebar to [view a list](#alert-management-list) of alerts. +There are several ways to accept alerts into your GitLab project, as outlined below. +Enabling any of these methods will allow the Alerts list to display. After configuring +alerts, visit **{cloud-gear}** **Operations > Alerts** in your project's sidebar +to [view the list](#alert-management-list) of alerts. - +### Opsgenie integration **(PREMIUM)** -## Populate Alert data +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. -To populate data, see the instructions for -[customizing the payload](../integrations/generic_alerts.md) and making a -request to the alerts endpoint. +A new way of monitoring Alerts via a GitLab integration is with +[Opsgenie](https://www.atlassian.com/software/opsgenie). + +NOTE: **Note:** +If you enable the Opsgenie integration, you cannot have other GitLab alert services, +such as [Generic Alerts](../integrations/generic_alerts.md) or +Prometheus alerts, active at the same time. + +To enable Opsgenie integration: + +1. Sign in as a user with Maintainer or Owner [permissions](../../permissions.md). +1. Navigate to **{cloud-gear}** **Operations > Alerts**. +1. In the **Integrations** select box, select Opsgenie. +1. Click the **Active** toggle. +1. In the **API URL**, enter the base URL for your Opsgenie integration, such + as `https://app.opsgenie.com/alert/list`. +1. Click **Save changes**. + +After enabling the integration, navigate to the Alerts list page at **{cloud-gear}** **Operations > Alerts**, and click **View alerts in Opsgenie**. + +### Enable a Generic Alerts endpoint + +GitLab provides the Generic Alerts endpoint so you can accept alerts from a third-party +alerts service. See the +[instructions for toggling generic alerts](../integrations/generic_alerts.md#setting-up-generic-alerts) +to add this option. After configuring the endpoint, the +[Alerts list](#alert-management-list) is enabled. + +To populate the alerts with data, see [Customizing the payload](../integrations/generic_alerts.md#customizing-the-payload) for requests to the alerts endpoint. + +### Enable GitLab-managed Prometheus alerts + +You can install the GitLab-managed Prometheus application on your Kubernetes +cluster. For more information, see +[Managed Prometheus on Kubernetes](../integrations/prometheus.md#managed-prometheus-on-kubernetes). +When GitLab-managed Prometheus is installed, the [Alerts list](#alert-management-list) +is also enabled. + +To populate the alerts with data, see +[GitLab-Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances). + +### Enable external Prometheus alerts + +You can configure an externally-managed Prometheus instance to send alerts +to GitLab. To set up this configuration, see the [configuring Prometheus](../../../operations/metrics/alerts.md#external-prometheus-instances) documentation. Activating the external Prometheus +configuration also enables the [Alerts list](#alert-management-list). + +To populate the alerts with data, see +[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances). ## Alert Management severity @@ -55,15 +103,42 @@ You will need at least Developer [permissions](../../permissions.md) to view the You can find the Alert Management list at **{cloud-gear}** **Operations > Alerts** in your project's sidebar. Each alert contains the following metrics: - + - **Severity** - The current importance of a alert and how much attention it should receive. - **Start time** - How long ago the alert fired. This field uses the standard GitLab pattern of `X time ago`, but is supported by a granular date/time tooltip depending on the user's locale. -- **End time** - How long ago the alert fired was resolved. This field uses the standard GitLab pattern of `X time ago`, but is supported by a granular date/time tooltip depending on the user's locale. - **Alert description** - The description of the alert, which attempts to capture the most meaningful data. - **Event count** - The number of times that an alert has fired. +- **Issue** - A link to the incident issue that has been created for the alert. - **Status** - The [current status](#alert-management-statuses) of the alert. +### Alert Management list sorting + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217745) in GitLab 13.1. + +The Alert Management list displays alerts sorted by start time, but you can +change the sort order by clicking the headers in the Alert Management list. + +To see if a column is sortable, point your mouse at the header. Sortable columns +display an arrow next to the column name, as shown in this example: + + + +### Searching alerts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213884) in GitLab 13.1. + +The alert list supports a simple free text search. + + + +This search filters on the following fields: + +- Title +- Description +- Monitoring tool +- Service + ### Alert Management statuses Each alert contains a status dropdown to indicate which alerts need investigation. @@ -138,14 +213,43 @@ deselect the user from the list of assignees, or click **Unassigned**. > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab 13.1. -NOTE: **Note:** -GitLab currently only supports creating system notes when assigning an Alert. +When you take action on an alert, this is logged as a system note, +which is visible in the Alert Details view. This gives you a linear +timeline of the alert's investigation and assignment history. -Assigning a user an Alert creates a system note, visible in the Alert Details view, -giving you a linear timeline of the alert's investigation and assignment history. +The following actions will result in a system note: + +- [Updating the status of an alert](#update-an-alerts-status) +- [Creating an issue based on an alert](#create-an-issue-from-an-alert) +- [Assignment of an alert to a user](#update-an-alerts-assignee)  +### View an Alert's metrics data + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217768) in GitLab 13.2. + +To view the metrics for an alert: + + 1. Sign in as a user with Developer or higher [permissions](../../permissions.md). + 1. Navigate to **{cloud-gear}** **Operations > Alerts**. + 1. Click the alert you want to view. + 1. Below the title of the alert, click the **Metrics** tab. + + + +For GitLab-managed Prometheus instances, metrics data is automatically available +for the alert, making it easy to see surrounding behavior. See +[Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances) +for information on setting up alerts. + +For externally-managed Prometheus instances, you can configure your alerting rules to +display a chart in the alert. See +[Embedding metrics based on alerts in incident issues](../../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues) +for information on how to appropriately configure your alerting rules. See +[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances) +for information on setting up alerts for your self-managed Prometheus instance. + ## Use cases for assigning alerts Consider a team formed by different sections of monitoring, collaborating on a diff --git a/doc/user/project/operations/dashboard_settings.md b/doc/user/project/operations/dashboard_settings.md index e14c478ab7a..f30ce5024d6 100644 --- a/doc/user/project/operations/dashboard_settings.md +++ b/doc/user/project/operations/dashboard_settings.md @@ -9,6 +9,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can configure your [Monitoring dashboard](../integrations/prometheus.md) to display the time zone of your choice, and the links of your choice. +To configure these settings you must have Manage Project +Operations [permissions](../../permissions.md). + ## Change the dashboard time zone > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214370) in GitLab 13.1. @@ -17,6 +20,7 @@ By default, your monitoring dashboard displays dates and times in your local time zone, but you can display dates and times in UTC format. To change the time zone: +1. Sign in as a user with Manage Project Operations [permissions](../../permissions.md). 1. Navigate to **{settings}** **Settings > Operations**, and scroll to **Metrics Dashboard**. 1. In the **Dashboard timezone** select box, select *User's local timezone* @@ -32,6 +36,7 @@ time zone: You can add a button on your monitoring dashboard that links directly to your existing external dashboards: +1. Sign in as a user with Manage Project Operations [permissions](../../permissions.md). 1. Navigate to **{settings}** **Settings > Operations**, and scroll to **Metrics Dashboard**. 1. In **External dashboard URL**, provide the URL to your external dashboard: diff --git a/doc/user/project/operations/feature_flags.md b/doc/user/project/operations/feature_flags.md index a9729204cd7..b0f7cfc966a 100644 --- a/doc/user/project/operations/feature_flags.md +++ b/doc/user/project/operations/feature_flags.md @@ -1,261 +1,5 @@ --- -stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../../../operations/feature_flags.md' --- -# Feature Flags **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/7433) in GitLab 11.4. - -With Feature Flags, you can deploy your application's new features to production in smaller batches. -You can toggle a feature on and off to subsets of users, helping you achieve Continuous Delivery. -Feature flags help reduce risk, allowing you to do controlled testing, and separate feature -delivery from customer launch. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an example of feature flags in action, see [GitLab for Deploys, Feature Flags, and Error Tracking](https://www.youtube.com/embed/5tw2p6lwXxo). - -## How it works - -GitLab uses [Unleash](https://github.com/Unleash/unleash), a feature -toggle service. - -By enabling or disabling a flag in GitLab, your application -can determine which features to enable or disable. - -You can create feature flags in GitLab and use the API from your application -to get the list of feature flags and their statuses. The application must be configured to communicate -with GitLab, so it's up to developers to use a compatible client library and -[integrate the feature flags in your app](#integrate-feature-flags-with-your-application). - -## Create a feature flag - -To create and enable a feature flag: - -1. Navigate to your project's **Operations > Feature Flags**. -1. Click the **New feature flag** button. -1. Enter a name that starts with a letter and contains only lowercase letters, digits, underscores (`_`) - and dashes (`-`), and does not end with a dash (`-`) or underscore (`_`). -1. Enter a description (optional, 255 characters max). -1. Enter details about how the flag should be applied: - - In GitLab 13.0 and earlier: Enter an environment spec, - enable or disable the flag in this environment, and select a rollout strategy. - - In GitLab 13.1 and later (when [this feature flag](#feature-flag-behavior-change-in-130) is enabled): Select a strategy and then - the environments to apply the strategy to. -1. Click **Create feature flag**. - -The feature flag is displayed in the list. It is enabled by default. - -## Disable a feature flag for a specific environment - -In [GitLab 13.0 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/8621), -to disable a feature flag for a specific environment: - -1. Navigate to your project's **Operations > Feature Flags**. -1. For the feature flag you want to disable, click the Pencil icon. -1. To disable the flag: - - In GitLab 13.0 and earlier: Slide the Status toggle for the environment. Or, to delete the - environment spec, on the right, click the **Remove (X)** icon. - - In GitLab 13.1 and later (when [this feature flag](#feature-flag-behavior-change-in-130) is - enabled): For each strategy it applies to, under **Environments**, delete the environment. -1. Click **Save changes**. - -## Disable a feature flag for all environments - -To disable a feature flag for all environments: - -1. Navigate to your project's **Operations > Feature Flags**. -1. For the feature flag you want to disable, slide the Status toggle to **Disabled**. - -The feature flag is displayed on the **Disabled** tab. - -## Feature flag behavior change in 13.0 - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35555) in GitLab 13.0. - -Starting in GitLab 13.0, you can apply a feature flag strategy across multiple environments, -without defining the strategy multiple times. - -This feature is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. - -To enable it: - -```ruby -Feature.enable(:feature_flags_new_version) -``` - -To disable it: - -```ruby -Feature.disable(:feature_flags_new_version) -``` - -## Feature flag strategies - -GitLab Feature Flag uses [Unleash](https://unleash.github.io) -as the feature flag engine. In Unleash, there is a concept of rulesets for granular feature flag controls, -called [strategies](https://unleash.github.io/docs/activation_strategy). -Supported strategies for GitLab Feature Flags are described below. - -### Rollout strategy - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8240) in GitLab 12.2. - -The selected rollout strategy affects which users will experience the feature as enabled. - -The status of an environment spec ultimately determines whether or not a feature is enabled at all. -For instance, a feature will always be disabled for every user if the matching environment spec has a disabled status, regardless of the chosen rollout strategy. -However, a feature will be enabled for 50% of logged-in users if the matching environment spec has an enabled status along with a **Percent rollout (logged in users)** strategy set to 50%. - -#### All users - -Enables the feature for all users. It is implemented using the Unleash -[`default`](https://unleash.github.io/docs/activation_strategy#default) -activation strategy. - -#### Percent rollout (logged in users) - -Enables the feature for a percentage of authenticated users. It is -implemented using the Unleash -[`gradualRolloutUserId`](https://unleash.github.io/docs/activation_strategy#gradualrolloutuserid) -activation strategy. - -Set a value of 15%, for example, to enable the feature for 15% of authenticated users. - -A rollout percentage may be between 0% and 100%. - -CAUTION: **Caution:** -If this strategy is selected, then the Unleash client **must** be given a user -ID for the feature to be enabled. See the [Ruby example](#ruby-application-example) below. - -#### User IDs - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8240) in GitLab 12.2. [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/34363) to be defined per environment in GitLab 12.6. - -A feature flag may be enabled for a list of target users. It is implemented -using the Unleash [`userWithId`](https://unleash.github.io/docs/activation_strategy#userwithid) -activation strategy. - -User IDs should be a comma-separated list of values. For example, `user@example.com, user2@example.com`, or `username1,username2,username3`, etc. - -CAUTION: **Caution:** -The Unleash client **must** be given a user ID for the feature to be enabled for -target users. See the [Ruby example](#ruby-application-example) below. - -## Integrate feature flags with your application - -To use feature flags with your application, get access credentials from GitLab. -Then prepare your application with a client library. - -### Get access credentials - -To get the access credentials that your application needs to communicate with GitLab: - -1. Navigate to your project's **Operations > Feature Flags**. -1. Click the **Configure** button to view the following: - - **API URL**: URL where the client (application) connects to get a list of feature flags. - - **Instance ID**: Unique token that authorizes the retrieval of the feature flags. - - **Application name**: The name of the running environment. For instance, - if the application runs for a production server, the application name would be - `production` or similar. This value is used for the environment spec evaluation. - -NOTE: **Note:** -The meaning of these fields might change over time. For example, we are not sure -if **Instance ID** will be single token or multiple tokens, assigned to the -**Environment**. Also, **Application name** could describe the version of -application instead of the running environment. - -### Choose a client library - -GitLab implements a single backend that is compatible with Unleash clients. - -With the Unleash client, developers can define, in the application code, the default values for flags. -Each feature flag evaluation can express the desired outcome if the flag isn't present in the -provided configuration file. - -Unleash currently [offers many SDKs for various languages and frameworks](https://github.com/Unleash/unleash#client-implementations). - -### Feature flags API information - -For API content, see: - -- [Feature Flags API](../../../api/feature_flags.md) -- [Feature Flag Specs API](../../../api/feature_flag_specs.md) (Deprecated and [scheduled for removal in GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213369).) -- [Feature Flag User Lists API](../../../api/feature_flag_user_lists.md) -- [Legacy Feature Flags API](../../../api/feature_flags_legacy.md) - -### Golang application example - -Here's an example of how to integrate feature flags in a Golang application: - -```golang -package main - -import ( - "io" - "log" - "net/http" - - "github.com/Unleash/unleash-client-go" -) - -type metricsInterface struct { -} - -func init() { - unleash.Initialize( - unleash.WithUrl("https://gitlab.com/api/v4/feature_flags/unleash/42"), - unleash.WithInstanceId("29QmjsW6KngPR5JNPMWx"), - unleash.WithAppName("production"), - unleash.WithListener(&metricsInterface{}), - ) -} - -func helloServer(w http.ResponseWriter, req *http.Request) { - if unleash.IsEnabled("my_feature_name") { - io.WriteString(w, "Feature enabled\n") - } else { - io.WriteString(w, "hello, world!\n") - } -} - -func main() { - http.HandleFunc("/", helloServer) - log.Fatal(http.ListenAndServe(":8080", nil)) -} -``` - -### Ruby application example - -Here's an example of how to integrate feature flags in a Ruby application. - -The Unleash client is given a user ID for use with a **Percent rollout (logged in users)** rollout strategy or a list of **Target Users**. - -```ruby -#!/usr/bin/env ruby - -require 'unleash' -require 'unleash/context' - -unleash = Unleash::Client.new({ - url: 'http://gitlab.com/api/v4/feature_flags/unleash/42', - app_name: 'production', - instance_id: '29QmjsW6KngPR5JNPMWx' -}) - -unleash_context = Unleash::Context.new -# Replace "123" with the id of an authenticated user. -# Note that the context's user id must be a string: -# https://unleash.github.io/docs/unleash_context -unleash_context.user_id = "123" - -if unleash.is_enabled?("my_feature_name", unleash_context) - puts "Feature enabled" -else - puts "hello, world!" -end -``` +This document was moved to [another location](../../../operations/feature_flags.md). diff --git a/doc/user/project/operations/img/alert_detail_metrics_v13_2.png b/doc/user/project/operations/img/alert_detail_metrics_v13_2.png Binary files differnew file mode 100644 index 00000000000..84d83365ea8 --- /dev/null +++ b/doc/user/project/operations/img/alert_detail_metrics_v13_2.png diff --git a/doc/user/project/operations/img/alert_list_search_v13_1.png b/doc/user/project/operations/img/alert_list_search_v13_1.png Binary files differnew file mode 100644 index 00000000000..ba993fe530b --- /dev/null +++ b/doc/user/project/operations/img/alert_list_search_v13_1.png diff --git a/doc/user/project/operations/img/alert_list_sort_v13_1.png b/doc/user/project/operations/img/alert_list_sort_v13_1.png Binary files differnew file mode 100644 index 00000000000..8e06c3478f7 --- /dev/null +++ b/doc/user/project/operations/img/alert_list_sort_v13_1.png diff --git a/doc/user/project/operations/img/alert_list_v13_1.png b/doc/user/project/operations/img/alert_list_v13_1.png Binary files differnew file mode 100644 index 00000000000..7a1a5f5191e --- /dev/null +++ b/doc/user/project/operations/img/alert_list_v13_1.png diff --git a/doc/user/project/operations/img/alert_management_1_v13_0.png b/doc/user/project/operations/img/alert_management_1_v13_0.png Binary files differdeleted file mode 100644 index dbc1e795b16..00000000000 --- a/doc/user/project/operations/img/alert_management_1_v13_0.png +++ /dev/null diff --git a/doc/user/project/operations/img/alert_management_1_v13_1.png b/doc/user/project/operations/img/alert_management_1_v13_1.png Binary files differdeleted file mode 100644 index 00aa56a6050..00000000000 --- a/doc/user/project/operations/img/alert_management_1_v13_1.png +++ /dev/null diff --git a/doc/user/project/operations/index.md b/doc/user/project/operations/index.md index ca38eab9455..9cec578bc5e 100644 --- a/doc/user/project/operations/index.md +++ b/doc/user/project/operations/index.md @@ -1,13 +1,5 @@ -# Project operations +--- +redirect_to: '../../../operations/index.md' +--- -GitLab provides a variety of tools to help operate and maintain -your applications: - -- Collect [Prometheus metrics](../integrations/prometheus_library/index.md). -- Deploy to different [environments](../../../ci/environments/index.md). -- Connect your project to a [Kubernetes cluster](../clusters/index.md). -- Manage your infrastructure with [Infrastructure as Code](../../infrastructure/index.md) approaches. -- Discover and view errors generated by your applications with [Error Tracking](error_tracking.md). -- Create, toggle, and remove [Feature Flags](feature_flags.md). **(PREMIUM)** -- [Trace](tracing.md) the performance and health of a deployed application. **(ULTIMATE)** -- Change the [settings of the Monitoring Dashboard](dashboard_settings.md). +This document was moved to [another location](../../../operations/index.md). diff --git a/doc/user/project/operations/tracing.md b/doc/user/project/operations/tracing.md index 07f60c37f1b..87567f45560 100644 --- a/doc/user/project/operations/tracing.md +++ b/doc/user/project/operations/tracing.md @@ -1,40 +1,5 @@ --- -stage: Monitor -group: APM -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../../../operations/tracing.md' --- -# Tracing **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/7903) in GitLab Ultimate 11.5. - -Tracing provides insight into the performance and health of a deployed application, -tracking each function or microservice which handles a given request. - -This makes it easy to -understand the end-to-end flow of a request, regardless of whether you are using a monolithic or distributed system. - -## Jaeger tracing - -[Jaeger](https://www.jaegertracing.io/) is an open source, end-to-end distributed -tracing system used for monitoring and troubleshooting microservices-based distributed -systems. - -### Deploying Jaeger - -To learn more about deploying Jaeger, read the official -[Getting Started documentation](https://www.jaegertracing.io/docs/latest/getting-started/). -There is an easy to use [all-in-one Docker image](https://www.jaegertracing.io/docs/latest/getting-started/#AllinoneDockerimage), -as well as deployment options for [Kubernetes](https://github.com/jaegertracing/jaeger-kubernetes) -and [OpenShift](https://github.com/jaegertracing/jaeger-openshift). - -### Enabling Jaeger - -GitLab provides an easy way to open the Jaeger UI from within your project: - -1. [Set up Jaeger](https://www.jaegertracing.io) and configure your application using one of the - [client libraries](https://www.jaegertracing.io/docs/latest/client-libraries/). -1. Navigate to your project's **Settings > Operations** and provide the Jaeger URL. -1. Click **Save changes** for the changes to take effect. -1. You can now visit **Operations > Tracing** in your project's sidebar and - GitLab will redirect you to the configured Jaeger URL. +This document was moved to [another location](../../../operations/tracing.md). diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index bf9b58acf14..0425ca56285 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -287,7 +287,7 @@ To enable this setting: 1. Navigate to your project's **Settings > Pages**. 1. Tick the checkbox **Force HTTPS (requires valid certificates)**. -NOTE: **Note** +NOTE: **Note:** If you use CloudFlare CDN in front of GitLab Pages, make sure to set the SSL connection setting to `full` instead of `flexible`. For more details, see the [CloudFlare CDN directions](https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). <!-- ## Troubleshooting diff --git a/doc/user/project/pages/getting_started/fork_sample_project.md b/doc/user/project/pages/getting_started/fork_sample_project.md index de9bd97b262..250e90f0302 100644 --- a/doc/user/project/pages/getting_started/fork_sample_project.md +++ b/doc/user/project/pages/getting_started/fork_sample_project.md @@ -1,56 +1,5 @@ --- -type: reference, howto -stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: 'pages_forked_sample_project.md' --- -# Create a Pages website from a forked sample - -GitLab provides [sample projects for the most popular Static Site Generators](https://gitlab.com/pages). -You can fork one of the sample projects and run the CI/CD pipeline to generate a Pages website. - -Fork a sample project when you want to test GitLab Pages or start a new project that's already -configured to generate a Pages site. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a [video tutorial](https://www.youtube.com/watch?v=TWqh9MtT4Bg) of how this works. - -To fork a sample project and create a Pages website: - -1. View the sample projects by going to the [GitLab Pages examples](https://gitlab.com/pages) group. -1. Click the name of the project you want to [fork](../../../../gitlab-basics/fork-project.md). -1. In the top right, click the **Fork** button, and then choose a namespace to fork to. -1. Go to your project's **CI/CD > Pipelines** and click **Run pipeline**. - GitLab CI/CD builds and deploys your site. - -The site can take approximately 30 minutes to deploy. -When the pipeline is finished, go to **Settings > Pages** to find the link to your website from your project. - -For every change pushed to your repository, GitLab CI/CD runs a new pipeline -that immediately publishes your changes to the Pages site. - -You can take some **optional** further steps: - -- _Remove the fork relationship._ If you want to contribute to the project you forked from, - you can keep this relationship. Otherwise, go to your project's **Settings > General**, - expand **Advanced settings**, and scroll down to **Remove fork relationship**: - -  - -- _Change the URL to match your namespace._ If your Pages site is hosted on GitLab.com, - you can rename it to `<namespace>.gitlab.io`, where `<namespace>` is your GitLab namespace - (the one you chose when you forked the project). - - - Go to your project's **Settings > General** and expand **Advanced**. Scroll down to - **Change path** and change the path to `<namespace>.gitlab.io`. - - For example, if your project's URL is `gitlab.com/gitlab-tests/jekyll`, your namespace is - `gitlab-tests`. - - If you set the repository path to `gitlab-tests.gitlab.io`, - the resulting URL for your Pages website is `https://gitlab-tests.gitlab.io`. - -  - - - Now go to your SSG's configuration file and change the [base URL](../getting_started_part_one.md#urls-and-baseurls) - from `"project-name"` to `""`. The project name setting varies by SSG and may not be in the config file. +This document was moved to [pages_forked_sample_project.md](pages_forked_sample_project.md). diff --git a/doc/user/project/pages/getting_started/new_or_existing_website.md b/doc/user/project/pages/getting_started/new_or_existing_website.md index 5d7126ab22e..86f36447b93 100644 --- a/doc/user/project/pages/getting_started/new_or_existing_website.md +++ b/doc/user/project/pages/getting_started/new_or_existing_website.md @@ -1,49 +1,5 @@ --- -type: reference, howto -stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: 'pages_ci_cd_template.md' --- -# Create a Pages website by using a CI/CD template - -GitLab provides `.gitlab-ci.yml` templates for the most popular Static Site Generators (SSGs). -You can create your own `.gitlab-ci.yml` file from one of these templates, and run -the CI/CD pipeline to generate a Pages website. - -Use a `.gitlab-ci.yml` template when you have an existing project that you want to add a Pages site to. - -Your GitLab repository should contain files specific to an SSG, or plain HTML. -After you complete these steps, you may need to do additional -configuration for the Pages site to generate properly. - -1. In the left sidebar, click **Project overview**. -1. Click **Set up CI/CD**. - -  - - If this button is not available, CI/CD is already configured for - your project. You may want to browse the `.gitlab-ci.yml` files - [in these projects instead](https://gitlab.com/pages). - -1. From the **Apply a template** list, choose a template for the SSG you're using. - You can also choose plain HTML. - -  - - If you don't find a corresponding template, you can view the - [GitLab Pages group of sample projects](https://gitlab.com/pages). - These projects contain `.gitlab-ci.yml` files that you can modify for your needs. - You can also [learn how to write your own `.gitlab-ci.yml` - file for GitLab Pages](../getting_started_part_four.md). - -1. Save and commit the `.gitlab-ci.yml` file. - -If everything is configured correctly, the site can take approximately 30 minutes to deploy. - -You can watch the pipeline run by going to **CI / CD > Pipelines**. -When the pipeline is finished, go to **Settings > Pages** to find the link to -your Pages website. - -For every change pushed to your repository, GitLab CI/CD runs a new pipeline -that immediately publishes your changes to the Pages site. +This document was moved to [pages_ci_cd_template.md](pages_ci_cd_template.md). diff --git a/doc/user/project/pages/getting_started/pages_bundled_template.md b/doc/user/project/pages/getting_started/pages_bundled_template.md index cfa4e0910db..bc706105606 100644 --- a/doc/user/project/pages/getting_started/pages_bundled_template.md +++ b/doc/user/project/pages/getting_started/pages_bundled_template.md @@ -1,34 +1,5 @@ --- -type: reference, howto -stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: 'pages_new_project_template.md' --- -# Create a Pages website from a new project template - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47857) in GitLab 11.8. - -GitLab provides templates for the most popular Static Site Generators (SSGs). -You can create a new project from a template and run the CI/CD pipeline to generate a Pages website. - -Use a template when you want to test GitLab Pages or start a new project that's already -configured to generate a Pages site. - -1. From the top navigation, click the **+** button and select **New project**. -1. Select **Create from Template**. -1. Next to one of the templates starting with **Pages**, click **Use template**. - -  - -1. Complete the form and click **Create project**. -1. From the left sidebar, navigate to your project's **CI/CD > Pipelines** - and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your - site. - -The site can take approximately 30 minutes to deploy. -When the pipeline is finished, go to **Settings > Pages** to find the link to -your Pages website. - -For every change pushed to your repository, GitLab CI/CD runs a new pipeline -that immediately publishes your changes to the Pages site. +This document was moved to [pages_new_project_template.md](pages_new_project_template.md). diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md new file mode 100644 index 00000000000..906ffe43285 --- /dev/null +++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md @@ -0,0 +1,49 @@ +--- +type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Create a Pages website by using a CI/CD template + +GitLab provides `.gitlab-ci.yml` templates for the most popular Static Site Generators (SSGs). +You can create your own `.gitlab-ci.yml` file from one of these templates, and run +the CI/CD pipeline to generate a Pages website. + +Use a `.gitlab-ci.yml` template when you have an existing project that you want to add a Pages site to. + +Your GitLab repository should contain files specific to an SSG, or plain HTML. +After you complete these steps, you may need to do additional +configuration for the Pages site to generate properly. + +1. In the left sidebar, click **Project overview**. +1. Click **Set up CI/CD**. + +  + + If this button is not available, CI/CD is already configured for + your project. You may want to browse the `.gitlab-ci.yml` files + [in these projects instead](https://gitlab.com/pages). + +1. From the **Apply a template** list, choose a template for the SSG you're using. + You can also choose plain HTML. + +  + + If you don't find a corresponding template, you can view the + [GitLab Pages group of sample projects](https://gitlab.com/pages). + These projects contain `.gitlab-ci.yml` files that you can modify for your needs. + You can also [learn how to write your own `.gitlab-ci.yml` + file for GitLab Pages](pages_from_scratch.md). + +1. Save and commit the `.gitlab-ci.yml` file. + +If everything is configured correctly, the site can take approximately 30 minutes to deploy. + +You can watch the pipeline run by going to **CI / CD > Pipelines**. +When the pipeline is finished, go to **Settings > Pages** to find the link to +your Pages website. + +For every change pushed to your repository, GitLab CI/CD runs a new pipeline +that immediately publishes your changes to the Pages site. diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md new file mode 100644 index 00000000000..de9bd97b262 --- /dev/null +++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md @@ -0,0 +1,56 @@ +--- +type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Create a Pages website from a forked sample + +GitLab provides [sample projects for the most popular Static Site Generators](https://gitlab.com/pages). +You can fork one of the sample projects and run the CI/CD pipeline to generate a Pages website. + +Fork a sample project when you want to test GitLab Pages or start a new project that's already +configured to generate a Pages site. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a [video tutorial](https://www.youtube.com/watch?v=TWqh9MtT4Bg) of how this works. + +To fork a sample project and create a Pages website: + +1. View the sample projects by going to the [GitLab Pages examples](https://gitlab.com/pages) group. +1. Click the name of the project you want to [fork](../../../../gitlab-basics/fork-project.md). +1. In the top right, click the **Fork** button, and then choose a namespace to fork to. +1. Go to your project's **CI/CD > Pipelines** and click **Run pipeline**. + GitLab CI/CD builds and deploys your site. + +The site can take approximately 30 minutes to deploy. +When the pipeline is finished, go to **Settings > Pages** to find the link to your website from your project. + +For every change pushed to your repository, GitLab CI/CD runs a new pipeline +that immediately publishes your changes to the Pages site. + +You can take some **optional** further steps: + +- _Remove the fork relationship._ If you want to contribute to the project you forked from, + you can keep this relationship. Otherwise, go to your project's **Settings > General**, + expand **Advanced settings**, and scroll down to **Remove fork relationship**: + +  + +- _Change the URL to match your namespace._ If your Pages site is hosted on GitLab.com, + you can rename it to `<namespace>.gitlab.io`, where `<namespace>` is your GitLab namespace + (the one you chose when you forked the project). + + - Go to your project's **Settings > General** and expand **Advanced**. Scroll down to + **Change path** and change the path to `<namespace>.gitlab.io`. + + For example, if your project's URL is `gitlab.com/gitlab-tests/jekyll`, your namespace is + `gitlab-tests`. + + If you set the repository path to `gitlab-tests.gitlab.io`, + the resulting URL for your Pages website is `https://gitlab-tests.gitlab.io`. + +  + + - Now go to your SSG's configuration file and change the [base URL](../getting_started_part_one.md#urls-and-baseurls) + from `"project-name"` to `""`. The project name setting varies by SSG and may not be in the config file. diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md new file mode 100644 index 00000000000..7278c734b07 --- /dev/null +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -0,0 +1,402 @@ +--- +last_updated: 2020-01-06 +type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Create a GitLab Pages website from scratch + +This tutorial shows you how to create a Pages site from scratch. You will start with +a blank project and create your own CI file, which gives instruction to the +[GitLab Runner](https://docs.gitlab.com/runner/). When your CI/CD +[pipeline](../../../../ci/pipelines/index.md) runs, the Pages site is created. + +This example uses the [Jekyll](https://jekyllrb.com/) Static Site Generator (SSG). +Other SSGs follow similar steps. You do not need to be familiar with Jekyll or SSGs +to complete this tutorial. + +## Prerequisites + +To follow along with this example, start with a blank project in GitLab. +Create three files in the root (top-level) directory. + +- `.gitlab-ci.yml` A YAML file that contains the commands you want to run. + For now, leave the file's contents blank. + +- `index.html` An HTML file you can populate with whatever HTML content you'd like, + for example: + + ```html + <html> + <head> + <title>Home</title> + </head> + <body> + <h1>Hello World!</h1> + </body> + </html> + ``` + +- [`Gemfile`](https://bundler.io/gemfile.html) A file that describes dependencies for Ruby programs. + Populate it with this content: + + ```ruby + source "https://rubygems.org" + + gem "jekyll" + ``` + +## Choose a Docker image + +In this example, the Runner uses a [Docker image](../../../../ci/docker/using_docker_images.md) +to run scripts and deploy the site. + +This specific Ruby image is maintained on [DockerHub](https://hub.docker.com/_/ruby). + +Edit your `.gitlab-ci.yml` and add this text as the first line. + +```yaml +image: ruby:2.7 +``` + +If your SSG needs [NodeJS](https://nodejs.org/) to build, you must specify an +image that contains NodeJS as part of its file system. For example, for a +[Hexo](https://gitlab.com/pages/hexo) site, you can use `image: node:12.17.0`. + +## Install Jekyll + +To run [Jekyll](https://jekyllrb.com/) locally, you would open your terminal and: + +- Install [Bundler](https://bundler.io/) by running `gem install bundler`. +- Create `Gemfile.lock` by running `bundle install`. +- Install Jekyll by running `bundle exec jekyll build`. + +In the `.gitlab-ci.yml` file, this is written as: + +```yaml +script: + - gem install bundler + - bundle install + - bundle exec jekyll build +``` + +In addition, in the `.gitlab-ci.yml` file, each `script` is organized by a `job`. +A `job` includes the scripts and settings you want to apply to that specific +task. + +```yaml +job: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build +``` + +For GitLab Pages, this `job` has a specific name, called `pages`. +This setting tells the Runner you want the job to deploy your website +with GitLab Pages: + +```yaml +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build +``` + +## Specify the `public` directory for output + +Jekyll needs to know where to generate its output. +GitLab Pages only considers files in a directory called `public`. + +Jekyll uses destination (`-d`) to specify an output directory for the built website: + +```yaml +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public +``` + +## Specify the `public` directory for artifacts + +Now that Jekyll has output the files to the `public` directory, +the Runner needs to know where to get them. The artifacts are stored +in the `public` directory: + +```yaml +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public +``` + +Paste this into `.gitlab-ci.yml` file, so it now looks like this: + +```yaml +image: ruby:2.7 + +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public +``` + +Now save and commit the `.gitlab-ci.yml` file. You can watch the pipeline run +by going to **CI / CD > Pipelines**. + +When it succeeds, go to **Settings > Pages** to view the URL where your site +is now available. + +If you want to do more advanced tasks, you can update your `.gitlab-ci.yml` file +with [any of the available settings](../../../../ci/yaml/README.md). You can check +your CI syntax with the [GitLab CI/CD Lint Tool](../../../../ci/yaml/README.md#validate-the-gitlab-ciyml). + +The following topics show other examples of other options you can add to your CI/CD file. + +## Deploy specific branches to a Pages site + +You may want to deploy to a Pages site only from specific branches. + +First, add a `workflow` section to force the pipeline to run only when changes are +pushed to branches: + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public +``` + +Then configure the pipeline to run the job for the master branch only. + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +pages: + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public + rules: + - if: '$CI_COMMIT_BRANCH == "master"' +``` + +## Specify a stage to deploy + +There are three default stages for GitLab CI/CD: build, test, +and deploy. + +If you want to test your script and check the built site before deploying +to production, you can run the test exactly as it will run when you +push to `master`. + +To specify a stage for your job to run in, +add a `stage` line to your CI file: + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +pages: + stage: deploy + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public + rules: + - if: '$CI_COMMIT_BRANCH == "master"' +``` + +Now add another job to the CI file, telling it to +test every push to every branch **except** the `master` branch: + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +pages: + stage: deploy + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d public + artifacts: + paths: + - public + rules: + - if: '$CI_COMMIT_BRANCH == "master"' + +test: + stage: test + script: + - gem install bundler + - bundle install + - bundle exec jekyll build -d test + artifacts: + paths: + - test + rules: + - if: '$CI_COMMIT_BRANCH != "master"' +``` + +When the `test` job runs in the `test` stage, Jekyll +builds the site in a directory called `test`. The job affects +all branches except `master`. + +When you apply stages to different jobs, every job in the same +stage builds in parallel. If your web application needs more than +one test before being deployed, you can run all your tests at the +same time. + +## Remove duplicate commands + +To avoid duplicating the same scripts in every job, you can add them +to a `before_script` section. + +In the example, `gem install bundler` and `bundle install` were running +for both jobs, `pages` and `test`. + +Move these commands to a `before_script` section: + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +before_script: + - gem install bundler + - bundle install + +pages: + stage: deploy + script: + - bundle exec jekyll build -d public + artifacts: + paths: + - public + rules: + - if: '$CI_COMMIT_BRANCH == "master"' + +test: + stage: test + script: + - bundle exec jekyll build -d test + artifacts: + paths: + - test + rules: + - if: '$CI_COMMIT_BRANCH != "master"' +``` + +## Build faster with cached dependencies + +To build faster, you can cache the installation files for your +project's dependencies by using the `cache` parameter. + +This example caches Jekyll dependencies in a `vendor` directory +when you run `bundle install`: + +```yaml +image: ruby:2.7 + +workflow: + rules: + - if: '$CI_COMMIT_BRANCH' + +cache: + paths: + - vendor/ + +before_script: + - gem install bundler + - bundle install --path vendor + +pages: + stage: deploy + script: + - bundle exec jekyll build -d public + artifacts: + paths: + - public + rules: + - if: '$CI_COMMIT_BRANCH == "master"' + +test: + stage: test + script: + - bundle exec jekyll build -d test + artifacts: + paths: + - test + rules: + - if: '$CI_COMMIT_BRANCH != "master"' +``` + +In this case, you need to exclude the `/vendor` +directory from the list of folders Jekyll builds. Otherwise, Jekyll +will try to build the directory contents along with the site. + +In the root directory, create a file called `_config.yml` +and add this content: + +```yaml +exclude: + - vendor +``` + +Now GitLab CI/CD not only builds the website, +but also pushes with **continuous tests** to feature-branches, +**caches** dependencies installed with Bundler, and +**continuously deploys** every push to the `master` branch. + +## Related topics + +For more information, see the following blog posts. + +- [Use GitLab CI/CD `environments` to deploy your + web app to staging and production](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/). +- Learn [how to run jobs sequentially, + in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2016/07/29/the-basics-of-gitlab-ci/). +- Learn [how to pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) + to deploy this website, <https://docs.gitlab.com>. +- Learn [how to use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/). diff --git a/doc/user/project/pages/getting_started/pages_new_project_template.md b/doc/user/project/pages/getting_started/pages_new_project_template.md new file mode 100644 index 00000000000..cfa4e0910db --- /dev/null +++ b/doc/user/project/pages/getting_started/pages_new_project_template.md @@ -0,0 +1,34 @@ +--- +type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Create a Pages website from a new project template + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47857) in GitLab 11.8. + +GitLab provides templates for the most popular Static Site Generators (SSGs). +You can create a new project from a template and run the CI/CD pipeline to generate a Pages website. + +Use a template when you want to test GitLab Pages or start a new project that's already +configured to generate a Pages site. + +1. From the top navigation, click the **+** button and select **New project**. +1. Select **Create from Template**. +1. Next to one of the templates starting with **Pages**, click **Use template**. + +  + +1. Complete the form and click **Create project**. +1. From the left sidebar, navigate to your project's **CI/CD > Pipelines** + and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your + site. + +The site can take approximately 30 minutes to deploy. +When the pipeline is finished, go to **Settings > Pages** to find the link to +your Pages website. + +For every change pushed to your repository, GitLab CI/CD runs a new pipeline +that immediately publishes your changes to the Pages site. diff --git a/doc/user/project/pages/getting_started_part_four.md b/doc/user/project/pages/getting_started_part_four.md index 8cf58280b88..e45befe004e 100644 --- a/doc/user/project/pages/getting_started_part_four.md +++ b/doc/user/project/pages/getting_started_part_four.md @@ -1,402 +1,5 @@ --- -last_updated: 2020-01-06 -type: reference, howto -stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: 'getting_started/pages_from_scratch.md' --- -# Create a GitLab Pages website from scratch - -This tutorial shows you how to create a Pages site from scratch. You will start with -a blank project and create your own CI file, which gives instruction to the -[GitLab Runner](https://docs.gitlab.com/runner/). When your CI/CD -[pipeline](../../../ci/pipelines/index.md) runs, the Pages site is created. - -This example uses the [Jekyll](https://jekyllrb.com/) Static Site Generator (SSG). -Other SSGs follow similar steps. You do not need to be familiar with Jekyll or SSGs -to complete this tutorial. - -## Prerequisites - -To follow along with this example, start with a blank project in GitLab. -Create three files in the root (top-level) directory. - -- `.gitlab-ci.yml` A YAML file that contains the commands you want to run. - For now, leave the file's contents blank. - -- `index.html` An HTML file you can populate with whatever HTML content you'd like, - for example: - - ```html - <html> - <head> - <title>Home</title> - </head> - <body> - <h1>Hello World!</h1> - </body> - </html> - ``` - -- [`Gemfile`](https://bundler.io/gemfile.html) A file that describes dependencies for Ruby programs. - Populate it with this content: - - ```ruby - source "https://rubygems.org" - - gem "jekyll" - ``` - -## Choose a Docker image - -In this example, the Runner uses a [Docker image](../../../ci/docker/using_docker_images.md) -to run scripts and deploy the site. - -This specific Ruby image is maintained on [DockerHub](https://hub.docker.com/_/ruby). - -Edit your `.gitlab-ci.yml` and add this text as the first line. - -```yaml -image: ruby:2.7 -``` - -If your SSG needs [NodeJS](https://nodejs.org/) to build, you must specify an -image that contains NodeJS as part of its file system. For example, for a -[Hexo](https://gitlab.com/pages/hexo) site, you can use `image: node:12.17.0`. - -## Install Jekyll - -To run [Jekyll](https://jekyllrb.com/) locally, you would open your terminal and: - -- Install [Bundler](https://bundler.io/) by running `gem install bundler`. -- Create `Gemfile.lock` by running `bundle install`. -- Install Jekyll by running `bundle exec jekyll build`. - -In the `.gitlab-ci.yml` file, this is written as: - -```yaml -script: - - gem install bundler - - bundle install - - bundle exec jekyll build -``` - -In addition, in the `.gitlab-ci.yml` file, each `script` is organized by a `job`. -A `job` includes the scripts and settings you want to apply to that specific -task. - -```yaml -job: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -``` - -For GitLab Pages, this `job` has a specific name, called `pages`. -This setting tells the Runner you want the job to deploy your website -with GitLab Pages: - -```yaml -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -``` - -## Specify the `public` directory for output - -Jekyll needs to know where to generate its output. -GitLab Pages only considers files in a directory called `public`. - -Jekyll uses destination (`-d`) to specify an output directory for the built website: - -```yaml -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public -``` - -## Specify the `public` directory for artifacts - -Now that Jekyll has output the files to the `public` directory, -the Runner needs to know where to get them. The artifacts are stored -in the `public` directory: - -```yaml -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public -``` - -Paste this into `.gitlab-ci.yml` file, so it now looks like this: - -```yaml -image: ruby:2.7 - -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public -``` - -Now save and commit the `.gitlab-ci.yml` file. You can watch the pipeline run -by going to **CI / CD > Pipelines**. - -When it succeeds, go to **Settings > Pages** to view the URL where your site -is now available. - -If you want to do more advanced tasks, you can update your `.gitlab-ci.yml` file -with [any of the available settings](../../../ci/yaml/README.md). You can check -your CI syntax with the [GitLab CI/CD Lint Tool](https://gitlab.com/ci/lint). - -The following topics show other examples of other options you can add to your CI/CD file. - -## Deploy specific branches to a Pages site - -You may want to deploy to a Pages site only from specific branches. - -First, add a `workflow` section to force the pipeline to run only when changes are -pushed to branches: - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public -``` - -Then configure the pipeline to run the job for the master branch only. - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -pages: - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public - rules: - - if: '$CI_COMMIT_BRANCH == "master"' -``` - -## Specify a stage to deploy - -There are three default stages for GitLab CI/CD: build, test, -and deploy. - -If you want to test your script and check the built site before deploying -to production, you can run the test exactly as it will run when you -push to `master`. - -To specify a stage for your job to run in, -add a `stage` line to your CI file: - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -pages: - stage: deploy - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public - rules: - - if: '$CI_COMMIT_BRANCH == "master"' -``` - -Now add another job to the CI file, telling it to -test every push to every branch **except** the `master` branch: - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -pages: - stage: deploy - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d public - artifacts: - paths: - - public - rules: - - if: '$CI_COMMIT_BRANCH == "master"' - -test: - stage: test - script: - - gem install bundler - - bundle install - - bundle exec jekyll build -d test - artifacts: - paths: - - test - rules: - - if: '$CI_COMMIT_BRANCH != "master"' -``` - -When the `test` job runs in the `test` stage, Jekyll -builds the site in a directory called `test`. The job affects -all branches except `master`. - -When you apply stages to different jobs, every job in the same -stage builds in parallel. If your web application needs more than -one test before being deployed, you can run all your tests at the -same time. - -## Remove duplicate commands - -To avoid duplicating the same scripts in every job, you can add them -to a `before_script` section. - -In the example, `gem install bundler` and `bundle install` were running -for both jobs, `pages` and `test`. - -Move these commands to a `before_script` section: - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -before_script: - - gem install bundler - - bundle install - -pages: - stage: deploy - script: - - bundle exec jekyll build -d public - artifacts: - paths: - - public - rules: - - if: '$CI_COMMIT_BRANCH == "master"' - -test: - stage: test - script: - - bundle exec jekyll build -d test - artifacts: - paths: - - test - rules: - - if: '$CI_COMMIT_BRANCH != "master"' -``` - -## Build faster with cached dependencies - -To build faster, you can cache the installation files for your -project's dependencies by using the `cache` parameter. - -This example caches Jekyll dependencies in a `vendor` directory -when you run `bundle install`: - -```yaml -image: ruby:2.7 - -workflow: - rules: - - if: '$CI_COMMIT_BRANCH' - -cache: - paths: - - vendor/ - -before_script: - - gem install bundler - - bundle install --path vendor - -pages: - stage: deploy - script: - - bundle exec jekyll build -d public - artifacts: - paths: - - public - rules: - - if: '$CI_COMMIT_BRANCH == "master"' - -test: - stage: test - script: - - bundle exec jekyll build -d test - artifacts: - paths: - - test - rules: - - if: '$CI_COMMIT_BRANCH != "master"' -``` - -In this case, you need to exclude the `/vendor` -directory from the list of folders Jekyll builds. Otherwise, Jekyll -will try to build the directory contents along with the site. - -In the root directory, create a file called `_config.yml` -and add this content: - -```yaml -exclude: - - vendor -``` - -Now GitLab CI/CD not only builds the website, -but also pushes with **continuous tests** to feature-branches, -**caches** dependencies installed with Bundler, and -**continuously deploys** every push to the `master` branch. - -## Related topics - -For more information, see the following blog posts. - -- [Use GitLab CI/CD `environments` to deploy your - web app to staging and production](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/). -- Learn [how to run jobs sequentially, - in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2016/07/29/the-basics-of-gitlab-ci/). -- Learn [how to pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) - to deploy this website, <https://docs.gitlab.com>. -- Learn [how to use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/). +This document was moved to [getting_started/pages_from_scratch.md](getting_started/pages_from_scratch.md). diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 5861282ca6f..b116c28d94b 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -46,10 +46,10 @@ To create a GitLab Pages website: | Document | Description | | -------- | ----------- | -| [Use a `.gitlab-ci.yml` template](getting_started/new_or_existing_website.md) | Add a Pages site to an existing project. Use a pre-populated CI template file. | -| [Create a `gitlab-ci.yml` file from scratch](getting_started_part_four.md) | Add a Pages site to an existing project. Learn how to create and configure your own CI file. | -| [Use a new project template](getting_started/pages_bundled_template.md) | Create a new project with Pages already configured by using a new project template. | -| [Fork a sample project](getting_started/fork_sample_project.md) | Create a new project with Pages already configured by forking a sample project. | +| [Fork a sample project](getting_started/pages_forked_sample_project.md) | Create a new project with Pages already configured by forking a sample project. | +| [Use a new project template](getting_started/pages_new_project_template.md) | Create a new project with Pages already configured by using a new project template. | +| [Use a `.gitlab-ci.yml` template](getting_started/pages_ci_cd_template.md) | Add a Pages site to an existing project. Use a pre-populated CI template file. | +| [Create a `gitlab-ci.yml` file from scratch](getting_started/pages_from_scratch.md) | Add a Pages site to an existing project. Learn how to create and configure your own CI file. | To update a GitLab Pages website: @@ -81,7 +81,7 @@ becomes available automatically. To deploy your site, GitLab uses its built-in tool called [GitLab CI/CD](../../../ci/README.md) to build your site and publish it to the GitLab Pages server. The sequence of scripts that GitLab CI/CD runs to accomplish this task is created from a file named -`.gitlab-ci.yml`, which you can [create and modify](getting_started_part_four.md) at will. A specific `job` called `pages` in the configuration file will make GitLab aware that you are deploying a GitLab Pages website. +`.gitlab-ci.yml`, which you can [create and modify](getting_started/pages_from_scratch.md) at will. A specific `job` called `pages` in the configuration file will make GitLab aware that you are deploying a GitLab Pages website. You can either use GitLab's [default domain for GitLab Pages websites](getting_started_part_one.md#gitlab-pages-default-domain-names), `*.gitlab.io`, or your own domain (`example.com`). In that case, you'll diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 614a0d0dd19..a6923779f24 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -118,19 +118,19 @@ is so `cp` doesn't also copy `public/` to itself in an infinite loop: ```yaml pages: script: - - mkdir .public - - cp -r * .public - - mv .public public + - mkdir .public + - cp -r * .public + - mv .public public artifacts: paths: - - public + - public only: - - master + - master ``` ### `.gitlab-ci.yml` for a static site generator -See this document for a [step-by-step guide](getting_started_part_four.md). +See this document for a [step-by-step guide](getting_started/pages_from_scratch.md). ### `.gitlab-ci.yml` for a repository where there's also actual code @@ -161,13 +161,13 @@ image: ruby:2.6 pages: script: - - gem install jekyll - - jekyll build -d public/ + - gem install jekyll + - jekyll build -d public/ artifacts: paths: - - public + - public only: - - pages + - pages ``` See an example that has different files in the [`master` branch](https://gitlab.com/pages/jekyll-branched/tree/master) diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index e80b8098bec..bb5bca39398 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -12,7 +12,7 @@ This feature evolved out of [protected branches](protected_branches.md) ## Overview -Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Maintainer permission will be prevented from creating tags. +Protected tags will prevent anyone from updating or deleting the tag, and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Maintainer permission will be prevented from creating tags. ## Configuring protected tags diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index a3df173bd9d..def05bf94e4 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -7,14 +7,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Quick Actions +> - Introduced in [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26672): +> once an action is executed, an alert appears when a quick action is successfully applied. +> - In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/16877) and later, you can use +> quick actions when updating the description of issues, epics, and merge requests. + Quick actions are textual shortcuts for common actions on issues, epics, merge requests, and commits that are usually done by clicking buttons or dropdowns in GitLab's UI. -You can enter these commands while creating a new issue or merge request, or -in comments of issues, epics, merge requests, and commits. Each command should be -on a separate line in order to be properly detected and executed. - -> From [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26672), once an -> action is executed, an alert appears when a quick action is successfully applied. +You can enter these commands in the description or in comments of issues, epics, merge requests, and commits. +Each command should be on a separate line in order to be properly detected and executed. ## Quick Actions for issues, merge requests and epics @@ -40,7 +41,7 @@ The following quick actions are applicable to descriptions, discussions and thre | `/create_merge_request <branch name>` | ✓ | | | Create a new merge request starting from the current issue. | | `/done` | ✓ | ✓ | ✓ | Mark To-Do as done. | | `/due <date>` | ✓ | | | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | -| `/duplicate <#issue>` | ✓ | | | Mark this issue as a duplicate of another issue and mark them as related. **(STARTER)** | +| `/duplicate <#issue>` | ✓ | | | Close this issue and mark as a duplicate of another issue. **(CORE)** Also, mark both as related. **(STARTER)** | | `/epic <epic>` | ✓ | | | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. **(PREMIUM)** | | `/estimate <<W>w <DD>d <hh>h <mm>m>` | ✓ | ✓ | | Set time estimate. For example, `/estimate 1w 3d 2h 14m`. | | `/iteration *iteration:iteration` | ✓ | | | Set iteration ([Introduced in GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/196795)) **(STARTER)** | diff --git a/doc/user/project/releases/img/custom_notifications_dropdown_v12_5.png b/doc/user/project/releases/img/custom_notifications_dropdown_v12_5.png Binary files differdeleted file mode 100644 index 879599a71f5..00000000000 --- a/doc/user/project/releases/img/custom_notifications_dropdown_v12_5.png +++ /dev/null diff --git a/doc/user/project/releases/img/custom_notifications_new_release_v12_5.png b/doc/user/project/releases/img/custom_notifications_new_release_v12_5.png Binary files differdeleted file mode 100644 index d136aa710b2..00000000000 --- a/doc/user/project/releases/img/custom_notifications_new_release_v12_5.png +++ /dev/null diff --git a/doc/user/project/releases/img/edit_release_page_v13_0.png b/doc/user/project/releases/img/edit_release_page_v13_0.png Binary files differdeleted file mode 100644 index 1b4343019af..00000000000 --- a/doc/user/project/releases/img/edit_release_page_v13_0.png +++ /dev/null diff --git a/doc/user/project/releases/img/milestone_list_with_releases_v12_5.png b/doc/user/project/releases/img/milestone_list_with_releases_v12_5.png Binary files differindex 2e3ec08ba87..efe48058d9a 100644 --- a/doc/user/project/releases/img/milestone_list_with_releases_v12_5.png +++ b/doc/user/project/releases/img/milestone_list_with_releases_v12_5.png diff --git a/doc/user/project/releases/img/milestone_with_releases_v12_5.png b/doc/user/project/releases/img/milestone_with_releases_v12_5.png Binary files differdeleted file mode 100644 index 8719a58ce4e..00000000000 --- a/doc/user/project/releases/img/milestone_with_releases_v12_5.png +++ /dev/null diff --git a/doc/user/project/releases/img/new_tag_12_5.png b/doc/user/project/releases/img/new_tag_12_5.png Binary files differdeleted file mode 100644 index 9a6145d71c7..00000000000 --- a/doc/user/project/releases/img/new_tag_12_5.png +++ /dev/null diff --git a/doc/user/project/releases/img/release_edit_button_v12_6.png b/doc/user/project/releases/img/release_edit_button_v12_6.png Binary files differdeleted file mode 100644 index 8cc080621cf..00000000000 --- a/doc/user/project/releases/img/release_edit_button_v12_6.png +++ /dev/null diff --git a/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png b/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png Binary files differdeleted file mode 100644 index 453c7ca93cc..00000000000 --- a/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png +++ /dev/null diff --git a/doc/user/project/releases/img/release_with_milestone_v12_9.png b/doc/user/project/releases/img/release_with_milestone_v12_9.png Binary files differindex 53100e33955..c828e36114a 100644 --- a/doc/user/project/releases/img/release_with_milestone_v12_9.png +++ b/doc/user/project/releases/img/release_with_milestone_v12_9.png diff --git a/doc/user/project/releases/img/releases_count_v13_2.png b/doc/user/project/releases/img/releases_count_v13_2.png Binary files differnew file mode 100644 index 00000000000..1c0493326d1 --- /dev/null +++ b/doc/user/project/releases/img/releases_count_v13_2.png diff --git a/doc/user/project/releases/img/releases_v12_9.png b/doc/user/project/releases/img/releases_v12_9.png Binary files differdeleted file mode 100644 index bd23cf76651..00000000000 --- a/doc/user/project/releases/img/releases_v12_9.png +++ /dev/null diff --git a/doc/user/project/releases/img/tags_12_5.png b/doc/user/project/releases/img/tags_12_5.png Binary files differdeleted file mode 100644 index c9673a5232d..00000000000 --- a/doc/user/project/releases/img/tags_12_5.png +++ /dev/null diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 58d143fb32b..258601574ca 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -9,261 +9,302 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41766) in GitLab 11.7. -It is typical to create a [Git tag](../../../university/training/topics/tags.md) at -the moment of release to introduce a checkpoint in your source code -history, but in most cases your users will need compiled objects or other -assets output by your CI system to use them, not just the raw source -code. - -GitLab's **Releases** are a way to track deliverables in your project. Consider them -a snapshot in time of the source, build output, artifacts, and other metadata +To introduce a checkpoint in your source code history, you can assign a +[Git tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) at the moment of release. +However, in most cases, your users need more than just the raw source code. +They need compiled objects or other assets output by your CI/CD system. + +A GitLab *Release* is a snapshot of the source, build output, artifacts, and other metadata associated with a released version of your code. -## Getting started with Releases +You can create a GitLab release on any branch. When you create a release: -Start by giving a [description](#release-description) to the Release and -including its [assets](#release-assets), as follows. +- GitLab automatically archives source code and associates it with the release. +- GitLab automatically creates a JSON file that lists everything in the release, + so you can compare and audit releases. This file is called [release evidence](#release-evidence). +- You can add release notes and a message for the tag associated with the release. -## Release versioning +After you create a release, you can [associate milestones with it](#associate-milestones-with-a-release), +and attach [release assets](#release-assets), like runbooks or packages. -Release versions are manually assigned by the user in the Release title. GitLab uses [Semantic Versioning](https://semver.org/) for our releases, and we recommend you do too. Use `(Major).(Minor).(Patch)`, as detailed in the [GitLab Policy for Versioning](../../../policy/maintenance.md#versioning). +## View releases -For example, for GitLab version `10.5.7`: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36667) in GitLab 12.8. -- `10` represents the major version. The major release was `10.0.0`, but often referred to as `10.0`. -- `5` represents the minor version. The minor release was `10.5.0`, but often referred to as `10.5`. -- `7` represents the patch number. +To view a list of releases: -Any part of the version number can be multiple digits, for example, `13.10.11`. +- Go to **Project overview > Releases**, or + +- On the project's overview page, if at least one release exists, click the number of releases. + +  -### Release description + - On public projects, this number is visible to all users. + - On private projects, this number is visible to users with Reporter + [permissions](../../permissions.md#project-members-permissions) or higher. -Every Release has a description. You can add any text you like, but we recommend -including a changelog to describe the content of your release. This will allow -your users to quickly scan the differences between each one you publish. +## Create a release + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32812) in GitLab 12.9. Releases can be created directly in the GitLab UI. NOTE: **Note:** -[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and -Release descriptions are unrelated. Description supports [Markdown](../../markdown.md). +Only users with Developer permissions or higher can create releases. +Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). -### Release assets +You can create a release in the user interface, or by using the +[Releases API](../../../api/releases/index.md#create-a-release). +We recommend using the API to add release notes as one of the last steps in your CI/CD release pipeline. -You can currently add the following types of assets to each Release: +To create a new release through the GitLab UI: -- [Source code](#source-code): state of the repository at the time of the Release -- [Links](#links): to content such as built binaries or documentation +1. Navigate to **Project overview > Releases** and click the **New release** button. +1. In the [**Tag name**](#tag-name) box, enter a name. +1. In the **Create from** list, select the branch or enter a tag or commit SHA. +1. In the **Message** box, enter a message associated with the tag. +1. Optionally, in the [**Release notes**](#release-notes-description) + field, enter the release's description. You can use Markdown and drag and drop files to this field. + - If you leave this field empty, only a tag will be created. + - If you populate it, both a tag and a release will be created. +1. Click **Create tag**. -GitLab will support more asset types in the future, including objects such -as pre-built packages, compliance/security evidence, or container images. +If you created a release, you can view it at **Project overview > Releases**. +If you created a tag, you can view it at **Repository > Tags**. -#### Source code +You can now edit the release to [add milestones](#associate-milestones-with-a-release) +and [release assets](#release-assets). -GitLab automatically generate `zip`, `tar.gz`, `tar.bz2` and `tar` -archived source code from the given Git tag. These are read-only assets. +### Schedule a future release -#### Links +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38105) in GitLab 12.1. -A link is any URL which can point to whatever you like; documentation, built -binaries, or other related materials. These can be both internal or external -links from your GitLab instance. +You can create a release ahead of time by using the [Releases API](../../../api/releases/index.md#upcoming-releases). +When you set a future `released_at` date, an **Upcoming Release** badge is displayed next to the +release tag. When the `released_at` date and time has passed, the badge is automatically removed. -The four types of links are "Runbook," "Package," "Image," and "Other." + -#### Permanent links to Release assets +## Edit a release -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27300) in GitLab 12.9. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26016) in GitLab 12.6. Asset link editing was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9427) in GitLab 12.10. -The assets associated with a Release are accessible through a permanent URL. -GitLab will always redirect this URL to the actual asset -location, so even if the assets move to a different location, you can continue -to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link). +NOTE: **Note:** +Only users with Developer permissions or higher can edit releases. +Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). -Each asset has a name, a URL of the *actual* asset location, and optionally, a -`filepath` parameter, which, if you specify it, will create a URL pointing -to the asset for the Release. The format of the URL is: +To edit the details of a release: -```html -https://host/namespace/project/releases/:release/downloads/:filepath -``` +1. Navigate to **Project overview > Releases**. +1. In the top-right corner of the release you want to modify, click **Edit this release** (the pencil icon). +1. On the **Edit Release** page, change the release's details. +1. Click **Save changes**. -If you have an asset for the `v11.9.0-rc2` release in the `gitlab-org` -namespace and `gitlab-runner` project on `gitlab.com`, for example: +You can edit the release title, notes, associated milestones, and asset links. +To change other release information, such as the tag or release date, use the +[Releases API](../../../api/releases/index.md#update-a-release). -```json -{ - "name": "linux amd64", - "filepath": "/binaries/gitlab-runner-linux-amd64", - "url": "https://gitlab-runner-downloads.s3.amazonaws.com/v11.9.0-rc2/binaries/gitlab-runner-linux-amd64" -} -``` +## Add release notes to Git tags -This asset has a direct link of: +If you have an existing Git tag, you can add release notes to it. -```html -https://gitlab.com/gitlab-org/gitlab-runner/releases/v11.9.0-rc2/downloads/binaries/gitlab-runner-linux-amd64 -``` +You can do this in the user interface, or by using the [Releases API](../../../api/releases/index.md). +We recommend using the API to add release notes as one of the last steps in your CI/CD release pipeline. -The physical location of the asset can change at any time and the direct link will remain unchanged. +In the interface, to add release notes to a new Git tag: + +1. Navigate to your project's **Repository > Tags**. +1. Click **New tag**. +1. In the **Release notes** field, enter the release's description. + You can use Markdown and drag and drop files to this field. +1. Click **Create tag**. + +In the interface, to add release notes to an existing Git tag: + +1. Navigate to your project's **Repository > Tags**. +1. Click **Edit release notes** (the pencil icon). +1. In the **Release notes** field, enter the release's description. + You can use Markdown in this field, and drag and drop files to it. +1. Click **Save changes**. -### Releases associated with milestones +## Associate milestones with a release > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29020) in GitLab 12.5. > - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/39467) to edit milestones in the UI in GitLab 13.0. -Releases can optionally be associated with one or more -[project milestones](../milestones/index.md#project-milestones-and-group-milestones) -by including a `milestones` array in your requests to the -[Releases API](../../../api/releases/index.md#create-a-release) or by using the dropdown in the [Edit Release](#editing-a-release) page. +You can associate a release with one or more [project milestones](../milestones/index.md#project-milestones-and-group-milestones). + +You can do this in the user interface, or by including a `milestones` array in your request to +the [Releases API](../../../api/releases/index.md#create-a-release). + +In the user interface, to associate milestones to a release: - +1. Navigate to **Project overview > Releases**. +1. In the top-right corner of the release you want to modify, click **Edit this release** (the pencil icon). +1. From the **Milestones** list, select each milestone you want to associate. You can select multiple milestones. +1. Click **Save changes**. -Releases display this association with the **Milestone** indicator in the top -section of the Release block on the **Project overview > Releases** page, along -with some statistics about the issues in the milestone(s). +On the **Project overview > Releases** page, the **Milestone** is listed in the top +section, along with statistics about the issues in the milestones.  -Below is an example of milestones with no Releases, one Release, and two -Releases, respectively. +Releases are also visible on the **Issues > Milestones** page, and when you click a milestone +on this page. + +Here is an example of milestones with no releases, one release, and two releases, respectively.  -This relationship is also visible in the **Releases** section of the sidebar -when viewing a specific milestone. Below is an example of a milestone -associated with a large number of Releases. +## Get notified when a release is created - +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26001) in GitLab 12.4. -## Releases list +You can be notified by email when a new release is created for your project. -Navigate to **Project > Releases** in order to see the list of releases for a given -project. +To subscribe to notifications for releases: - +1. Navigate to **Project overview**. +1. Click **Notification setting** (the bell icon). +1. In the list, click **Custom**. +1. Select the **New release** check box. +1. Close the dialog box to save. -### Number of Releases +## Prevent unintentional releases by setting a deploy freeze -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36667) in GitLab 12.8. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29382) in GitLab 13.0. -The incremental number of Releases is displayed on the project's details page. When clicked, -it takes you to the list of Releases. +Prevent unintended production releases during a period of time you specify by +setting a [*deploy freeze* period](../../../ci/environments/deployment_safety.md). +Deploy freezes help reduce uncertainty and risk when automating deployments. - +Use the [Freeze Periods API](../../../api/freeze_periods.md) to set a `freeze_start` and a `freeze_end`, which +are defined as [crontab](https://crontab.guru/) entries. -For private projects, the number of Releases is displayed to users with Reporter -[permissions](../../permissions.md#project-members-permissions) or higher. For public projects, -it is displayed to every user regardless of their permission level. +If the job that's executing is within a freeze period, GitLab CI/CD creates an environment +variable named `$CI_DEPLOY_FREEZE`. -### Upcoming Releases +To prevent the deployment job from executing, create a `rules` entry in your +`gitlab-ci.yaml`, for example: -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38105) in GitLab 12.1. +```yaml +deploy_to_production: + stage: deploy + script: deploy_to_prod.sh + rules: + - if: $CI_DEPLOY_FREEZE == null +``` -A Release may be created ahead of time by specifying a future `released_at` date. Until -the `released_at` date and time is reached, an **Upcoming Release** badge will appear next to the -Release tag. Once the `released_at` date and time has passed, the badge is automatically removed. +If a project contains multiple freeze periods, all periods apply. If they overlap, the freeze covers the +complete overlapping period. - +For more information, see [Deployment safety](../../../ci/environments/deployment_safety.md). -## Creating a Release +## Release fields -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32812) in GitLab 12.9, Releases can be created directly through the GitLab Releases UI. +The following fields are available when you create or edit a release. -NOTE: **Note:** -Only users with Developer permissions or higher can create Releases. -Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). +### Tag name -To create a new Release through the GitLab UI: +The release tag name should include the release version. GitLab uses [Semantic Versioning](https://semver.org/) +for our releases, and we recommend you do too. Use `(Major).(Minor).(Patch)`, as detailed in the +[GitLab Policy for Versioning](../../../policy/maintenance.md#versioning). -1. Navigate to **Project overview > Releases** and click the **New release** button. -1. On the **New Tag** page, fill out the tag details. -1. Optionally, in the **Release notes** field, enter the Release's description. - If you leave this field empty, only a tag will be created. - If you populate it, both a tag and a Release will be created. -1. Click **Create tag**. +For example, for GitLab version `10.5.7`: -If you created a release, you can view it at **Project overview > Releases**. +- `10` represents the major version. The major release was `10.0.0`, but often referred to as `10.0`. +- `5` represents the minor version. The minor release was `10.5.0`, but often referred to as `10.5`. +- `7` represents the patch number. -You can also create a Release using the [Releases API](../../../api/releases/index.md#create-a-release): -we recommend doing this as one of the last steps in your CI/CD release pipeline. +Any part of the version number can be multiple digits, for example, `13.10.11`. -## Editing a release +### Release notes description -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26016) in GitLab 12.6. Asset link editing was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9427) in GitLab 12.10. +Every release has a description. You can add any text you like, but we recommend +including a changelog to describe the content of your release. This helps users +quickly scan the differences between each release you publish. -To edit the details of a release, navigate to **Project overview > Releases** and click -the edit button (pencil icon) in the top-right corner of the release you want to modify. +NOTE: **Note:** +[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and +Release note descriptions are unrelated. Description supports [Markdown](../../markdown.md). - +### Release assets -This will bring you to the **Edit Release** page, from which you can -change some of the release's details. +You can currently add the following types of assets to each release: - +- [Source code](#source-code) +- [Links](#links) -Currently, it is only possible to edit the release title, notes, associated milestones, and asset -links. To change other release information, such as its tag, or release date, use the [Releases -API](../../../api/releases/index.md#update-a-release). Editing this information -through the **Edit Release** page is planned for a future version of GitLab. +GitLab will support more asset types in the future, including objects such +as pre-built packages, compliance/security evidence, or container images. -## Notification for Releases +#### Permanent links to release assets -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26001) in GitLab 12.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27300) in GitLab 12.9. -You can be notified by email when a new Release is created for your project. +The assets associated with a release are accessible through a permanent URL. +GitLab will always redirect this URL to the actual asset +location, so even if the assets move to a different location, you can continue +to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link). -To subscribe to Release notifications: +Each asset has a name, a URL of the *actual* asset location, and optionally, a +`filepath` parameter, which, if you specify it, will create a URL pointing +to the asset for the Release. The format of the URL is: -1. Navigate to your **Project**'s landing page. -1. Click the bell icon (**Notification setting**). -1. Select **Custom** from the dropdown menu. -  -1. Select **New release**. -  +```plaintext +https://host/namespace/project/releases/:release/downloads/:filepath +``` -## Add release notes to Git tags +If you have an asset for the `v11.9.0-rc2` release in the `gitlab-org` +namespace and `gitlab-runner` project on `gitlab.com`, for example: -You can add release notes to any Git tag using the notes feature. Release notes -behave like any other Markdown form in GitLab so you can write text and -drag and drop files to it. Release notes are stored in GitLab's database. +```json +{ + "name": "linux amd64", + "filepath": "/binaries/gitlab-runner-linux-amd64", + "url": "https://gitlab-runner-downloads.s3.amazonaws.com/v11.9.0-rc2/binaries/gitlab-runner-linux-amd64" +} +``` + +This asset has a direct link of: -There are several ways to add release notes: +```plaintext +https://gitlab.com/gitlab-org/gitlab-runner/releases/v11.9.0-rc2/downloads/binaries/gitlab-runner-linux-amd64 +``` + +The physical location of the asset can change at any time and the direct link will remain unchanged. -- In the interface, when you create a new Git tag. -- In the interface, by adding a release note to an existing Git tag. -- Using the [Releases API](../../../api/releases/index.md): (we recommend doing this as one of the last - steps in your CI/CD release pipeline). +### Source code -To create a new tag, navigate to your project's **Repository > Tags** and -click **New tag**. From there, you can fill the form with all the information -about the release: +GitLab automatically generates `zip`, `tar.gz`, `tar.bz2` and `tar` +archived source code from the given Git tag. These are read-only assets. - +### Links -You can also edit an existing tag to add release notes: +A link is any URL which can point to whatever you like: documentation, built +binaries, or other related materials. These can be both internal or external +links from your GitLab instance. - +The four types of links are "Runbook," "Package," "Image," and "Other." -## Release Evidence +## Release evidence > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26019) in GitLab 12.6. Each time a release is created, GitLab takes a snapshot of data that's related to it. -This data is called Release Evidence. It includes linked milestones and issues, and -it is taken at moment the release is created. It provides a chain of custody and can -facilitate processes like external audits, for example. +This data is saved in a JSON file and called *release evidence*. It includes linked milestones +and issues and can facilitate internal processes like external audits. + +To access the release evidence, on the Releases page, click the link to the JSON file that's listed +under the **Evidence collection** heading. You can also [use the API](../../../api/releases/index.md#collect-release-evidence-premium-only) to -generate Release Evidence for an existing release. Because of this, [each release](#releases-list) -can have multiple Release Evidence snapshots. You can view the Release Evidence and -its details on the Release page. +generate release evidence for an existing release. Because of this, each release +can have multiple release evidence snapshots. You can view the release evidence and +its details on the Releases page. NOTE: **Note:** When the issue tracker is disabled, release evidence [cannot be downloaded](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). -Release Evidence is stored as a JSON object, so you can compare evidence by using -commonly-available tools. - -Here is an example of a Release Evidence object: +Here is an example of a release evidence object: ```json { @@ -313,94 +354,95 @@ Here is an example of a Release Evidence object: "created_at": "2019-04-17 15:45:12 UTC", "issues": [] } + ], + "report_artifacts": [ + { + "url":"https://gitlab.example.com/root/project-name/-/jobs/111/artifacts/download" + } ] } } ``` -### Diabling Release Evidence display **(CORE ONLY)** +### Collect release evidence **(PREMIUM ONLY)** -This feature comes with the `:release_evidence_collection` feature flag -enabled by default in GitLab self-managed instances. To turn it off, -ask a GitLab administrator with Rails console access to run the following -command: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. -```ruby -Feature.disable(:release_evidence_collection) -``` +When a release is created, release evidence is automatically collected. To initiate evidence collection any other time, use an [API call](../../../api/releases/index.md#collect-release-evidence-premium-only). You can collect release evidence multiple times for one release. -NOTE: **Note:** -Please note that Release Evidence's data is collected regardless of this -feature flag, which only enables or disables the display of the data on the -Releases page. +Evidence collection snapshots are visible on the Releases page, along with the timestamp the evidence was collected. -### Collect release evidence **(PREMIUM ONLY)** +### Include report artifacts as release evidence **(ULTIMATE ONLY)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32773) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. -Evidence collection can be initiated by using an [API call](../../../api/releases/index.md#collect-release-evidence-premium-only) at any time. Evidence snapshots are visible on -the Release page, along with the timestamp the Evidence was collected. +When you create a release, if [job artifacts](../../../ci/pipelines/job_artifacts.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. -### Schedule release evidence collection +Although job artifacts normally expire, artifacts included in release evidence do not expire. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23697) in GitLab 12.8. +To enable job artifact collection you need to specify both: -When the `released_at` date and time is not provided, the date and time of Release -creation is used. The Evidence collection background job is immediately executed. +1. [`artifacts:paths`](../../../ci/yaml/README.md#artifactspaths) +1. [`artifacts:reports`](../../../ci/pipelines/job_artifacts.md#artifactsreports) -If a future `released_at` is specified, the Release becomes an **Upcoming Release**. In this -case, the Evidence is scheduled to be collected at the `released_at` date and time, via a -background job. +```yaml +ruby: + script: + - gem install bundler + - bundle install + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml + artifacts: + paths: + - rspec.xml + reports: + junit: rspec.xml +``` -If a past `released_at` is used, no Evidence is collected for the Release. +If the pipeline ran successfully, when you create your release, the `rspec.xml` file is saved as release evidence. -## GitLab Releaser +NOTE: **Note:** +If you [schedule release evidence collection](#schedule-release-evidence-collection), some artifacts may already be expired by the time of evidence collection. To avoid this you can use the [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) keyword. Learn more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-releaser/-/merge_requests/6) in GitLab 12.10. +### Schedule release evidence collection -GitLab Releaser is a CLI tool for managing GitLab Releases from the command line or from -GitLab CI/CD's configuration file, `.gitlab-ci.yml`. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23697) in GitLab 12.8. -With it, you can create, update, modify, and delete Releases right through the -terminal. +In the API: -Read the [GitLab Releaser documentation](https://gitlab.com/gitlab-org/gitlab-releaser/-/tree/master/docs/index.md) -for details. +- If you specify a future `released_at` date, the release becomes an **Upcoming Release** + and the evidence is collected on the date of the release. You cannot collect + release evidence before then. +- If you use a past `released_at` date, no evidence is collected. +- If you do not specify a `released_at` date, release evidence is collected on the + date the release is created. -## Set a deploy freeze +### Disable release evidence display **(CORE ONLY)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29382) in GitLab 13.0. +The `:release_evidence_collection` feature flag is enabled by default in GitLab +self-managed instances. To turn it off, ask a GitLab administrator with Rails console +access to run the following command: -With a deploy freeze, you can prevent an unintended production release during a -period of time you specify, whether a company event or public holiday. You can -now rely on the enforcement of policies that are typically outside the scope of -GitLab to reduce uncertainty and risk when automating deployments. +```ruby +Feature.disable(:release_evidence_collection) +``` -Deploy freeze periods are set at the Project level, and may be created and -managed using the [Freeze Periods API](../../../api/freeze_periods.md). -Each Freeze Period has a `freeze_start` and a `freeze_end`, which are defined -as [crontab](https://crontab.guru/) entries. If a project contains multiple -freeze periods, all will apply, and should they overlap, the freeze covers the -complete overlapped period. +NOTE: **Note:** +Release evidence is collected regardless of this feature flag, +which only enables or disables the display of the data on the +Releases page. -During pipeline processing, GitLab CI creates an environment variable named -`$CI_DEPLOY_FREEZE` if the currently executing job is within a -Freeze Period. +## GitLab Releaser -To take advantage of this variable, create a `rules` entry in your -`gitlab-ci.yaml` to prevent the deployment job from executing. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-releaser/-/merge_requests/6) in GitLab 12.10. -For example: +GitLab Releaser is a CLI tool for managing GitLab Releases from the command line or from +GitLab's CI/CD configuration file, `.gitlab-ci.yml`. -```yaml -deploy_to_production: - stage: deploy - script: deploy_to_prod.sh - rules: - - if: $CI_DEPLOY_FREEZE == null -``` +With it, you can create, update, modify, and delete releases right through the +terminal. -For more information, see [Deployment safety](../../../ci/environments/deployment_safety.md). +Read the [GitLab Releaser documentation](https://gitlab.com/gitlab-org/gitlab-releaser/-/tree/master/docs/index.md) +for details. <!-- ## Troubleshooting diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 5fc6aa184bd..f94ca7ac106 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -41,17 +41,51 @@ See also: ## Default branch When you create a new [project](../../index.md), GitLab sets `master` as the default -branch for your project. You can choose another branch to be your project's +branch of the repository. You can choose another branch to be your project's default under your project's **Settings > Repository**. -The default branch is the branch affected by the -[issue closing pattern](../../issues/managing_issues.md#closing-issues-automatically), -which means that _an issue will be closed when a merge request is merged to -the **default branch**_. +When closing issues directly from merge requests through the [issue closing pattern](../../issues/managing_issues.md#closing-issues-automatically), +the target is the project's **default branch**. -The default branch is also protected against accidental deletion. Read through -the documentation on [protected branches](../../protected_branches.md#protected-branches) -to learn more. +The default branch is also initially [protected](../../protected_branches.md#protected-branches) +against accidental deletion and forced pushes. + +### Custom initial branch name **(CORE ONLY)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221013) in GitLab 13.2. +> - It's deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It cannot be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-custom-initial-branch-name-core-only). **(CORE ONLY)** + +By default, when you create a new project in GitLab, the initial branch is called `master`. +For self-managed instances, a GitLab administrator can customize the initial branch name to something +else. This way, every new project created from then on will start from the custom branch name rather than `master`. To do so: + +1. Go to the **{admin}** **Admin Area > Settings > Repository** and expand **Default initial + branch name**. +1. Change the default initial branch to a custom name of your choice. +1. **Save Changes**. + +#### Enable or disable custom initial branch name **(CORE ONLY)** + +Setting the default initial branch name is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../../administration/feature_flags.md) +can opt to disable it for your instance. + +To disable it: + +```ruby +Feature.disable(:global_default_branch_name) +``` + +To enable it: + +```ruby +Feature.enable(:global_default_branch_name) +``` ## Compare diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 48975b7864e..0cf375009a0 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -186,7 +186,8 @@ updated every 15 minutes at most, so may not reflect recent activity. The displa The project size may differ slightly from one instance to another due to compression, housekeeping, and other factors. -[Repository size limit](../../admin_area/settings/account_and_limit_settings.md) may be set by admins. GitLab.com's repository size limit [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). +[Repository size limit](../../admin_area/settings/account_and_limit_settings.md) may be set by admins. +GitLab.com's repository size limit [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). ## Contributors diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 124150c441a..baad5027703 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -25,11 +25,16 @@ Rewriting repository history is a destructive operation. Make sure to backup you you begin. The best way back up a repository is to [export the project](../settings/import_export.md#exporting-a-project-and-its-data). +NOTE: **Note:** +Git LFS files can only be removed by an Administrator using a +[Rake task](../../../raketasks/cleanup.md). Removal of this limitation +[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/223621). + ## Purge files from repository history To make cloning your project faster, rewrite branches and tags to remove unwanted files. -1. [Install `git filter-repo`](https://github.com/newren/git-filter-repo/blob/master/INSTALL.md) +1. [Install `git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/INSTALL.md) using a supported package manager or from source. 1. Clone a fresh copy of the repository using `--bare`: @@ -40,12 +45,25 @@ To make cloning your project faster, rewrite branches and tags to remove unwante 1. Using `git filter-repo`, purge any files from the history of your repository. - To purge all large files, the `--strip-blobs-bigger-than` option can be used: + To purge large files, the `--strip-blobs-bigger-than` option can be used: ```shell git filter-repo --strip-blobs-bigger-than 10M ``` + To purge large files stored using Git LFS, the `--blob--callback` option can + be used. The example below, uses the callback to read the file size from the + Git LFS pointer, and removes files larger than 10MB. + + ```shell + git filter-repo --blob-callback ' + if blob.data.startswith(b"version https://git-lfs.github.com/spec/v1"): + size_in_bytes = int.from_bytes(blob.data[124:], byteorder="big") + if size_in_bytes > 10*1000: + blob.skip() + ' + ``` + To purge specific large files by path, the `--path` and `--invert-paths` options can be combined: ```shell @@ -80,6 +98,12 @@ To make cloning your project faster, rewrite branches and tags to remove unwante [Protected tags](../protected_tags.md) will cause this to fail. To proceed, you must remove tag protection, push, and then re-enable protected tags. +1. Manually run [project housekeeping](../../../administration/housekeeping.md#manual-housekeeping) + +NOTE: **Note:** +Project statistics are cached for performance. You may need to wait 5-10 minutes +to see a reduction in storage utilization. + ## Purge files from GitLab storage To reduce the size of your repository in GitLab, you must remove GitLab internal references to @@ -103,7 +127,7 @@ cannot be fetched at all. However, these refs can be accessed from the Git bundle inside a project export. -1. [Install `git filter-repo`](https://github.com/newren/git-filter-repo/blob/master/INSTALL.md) +1. [Install `git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/INSTALL.md) using a supported package manager or from source. 1. Generate a fresh [export from the @@ -128,7 +152,7 @@ However, these refs can be accessed from the Git bundle inside a project export. trying to remove internal refs, we will rely on the `commit-map` produced by each run to tell us which internal refs to remove. - NOTE:**Note:** + NOTE: **Note:** `git filter-repo` creates a new `commit-map` file every run, and overwrite the `commit-map` from the previous run. You will need this file from **every** run. Do the next step every time you run `git filter-repo`. @@ -176,6 +200,7 @@ You will receive an email once it has completed. When using repository cleanup, note: +- Project statistics are cached. You may need to wait 5-10 minutes to see a reduction in storage utilization. - Housekeeping prunes loose objects older than 2 weeks. This means objects added in the last 2 weeks will not be removed immediately. If you have access to the [Gitaly](../../../administration/gitaly/index.md) server, you may run `git gc --prune=now` to diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index f75b083e6dc..bdf13100a6a 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -114,7 +114,7 @@ After the mirror is created, this option can currently only be modified via the To set up a mirror from GitLab to GitHub, you need to follow these steps: -1. Create a [GitHub personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) with the `public_repo` box checked. +1. Create a [GitHub personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with the `public_repo` box checked. 1. Fill in the **Git repository URL** field using this format: `https://<your_github_username>@github.com/<your_github_group>/<your_github_project>.git`. 1. Fill in **Password** field with your GitHub personal access token. 1. Click the **Mirror repository** button. @@ -125,10 +125,10 @@ The repository will push soon. To force a push, click the appropriate button. ## Setting up a push mirror to another GitLab instance with 2FA activated -1. On the destination GitLab instance, create a [personal access token](../../profile/personal_access_tokens.md) with `API` scope. +1. On the destination GitLab instance, create a [personal access token](../../profile/personal_access_tokens.md) with `write_repository` scope. 1. On the source GitLab instance: 1. Fill in the **Git repository URL** field using this format: `https://oauth2@<destination host>/<your_gitlab_group_or_name>/<your_gitlab_project>.git`. - 1. Fill in **Password** field with the GitLab personal access token created on the destination GitLab instance. + 1. Fill in the **Password** field with the GitLab personal access token created on the destination GitLab instance. 1. Click the **Mirror repository** button. ## Pulling from a remote repository **(STARTER)** @@ -231,7 +231,7 @@ those you expect. GitLab.com and other code hosting sites publish their fingerprints in the open for you to check: - [AWS CodeCommit](https://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints) -- [Bitbucket](https://confluence.atlassian.com/bitbucket/ssh-keys-935365775.html) +- [Bitbucket](https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/) - [GitHub](https://help.github.com/en/github/authenticating-to-github/githubs-ssh-key-fingerprints) - [GitLab.com](../../gitlab_com/index.md#ssh-host-keys-fingerprints) - [Launchpad](https://help.launchpad.net/SSHFingerprints) diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index d55d5c5c2d8..ffbad4e0989 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -25,9 +25,11 @@ For a commit or tag to be *verified* by GitLab: which is usually up to three years. - The signing time is equal or later then commit time. -NOTE: **Note:** Certificate revocation lists are checked on a daily basis via background worker. +NOTE: **Note:** +Certificate revocation lists are checked on a daily basis via background worker. -NOTE: **Note:** Self signed certificates without `authorityKeyIdentifier`, +NOTE: **Note:** +Self signed certificates without `authorityKeyIdentifier`, `subjectKeyIdentifier`, and `crlDistributionPoints` are not supported. We recommend using certificates from a PKI that are in line with [RFC 5280](https://tools.ietf.org/html/rfc5280). diff --git a/doc/user/project/requirements/img/requirement_archive_view_v12_10.png b/doc/user/project/requirements/img/requirement_archive_view_v12_10.png Binary files differdeleted file mode 100644 index b3a52caba6c..00000000000 --- a/doc/user/project/requirements/img/requirement_archive_view_v12_10.png +++ /dev/null diff --git a/doc/user/project/requirements/img/requirement_create_view_v12_10.png b/doc/user/project/requirements/img/requirement_create_view_v12_10.png Binary files differdeleted file mode 100644 index ecb08fe8a8b..00000000000 --- a/doc/user/project/requirements/img/requirement_create_view_v12_10.png +++ /dev/null diff --git a/doc/user/project/requirements/img/requirement_edit_view_v12_10.png b/doc/user/project/requirements/img/requirement_edit_view_v12_10.png Binary files differdeleted file mode 100644 index 5251e7eae1e..00000000000 --- a/doc/user/project/requirements/img/requirement_edit_view_v12_10.png +++ /dev/null diff --git a/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png b/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png Binary files differdeleted file mode 100644 index a5487b46894..00000000000 --- a/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png +++ /dev/null diff --git a/doc/user/project/requirements/img/requirements_archived_list_view_v13_1.png b/doc/user/project/requirements/img/requirements_archived_list_view_v13_1.png Binary files differnew file mode 100644 index 00000000000..aafc8543bae --- /dev/null +++ b/doc/user/project/requirements/img/requirements_archived_list_view_v13_1.png diff --git a/doc/user/project/requirements/img/requirements_list_v13_1.png b/doc/user/project/requirements/img/requirements_list_v13_1.png Binary files differnew file mode 100644 index 00000000000..3d2adfac074 --- /dev/null +++ b/doc/user/project/requirements/img/requirements_list_v13_1.png diff --git a/doc/user/project/requirements/img/requirements_list_view_v12_10.png b/doc/user/project/requirements/img/requirements_list_view_v12_10.png Binary files differdeleted file mode 100644 index cee1f3781f6..00000000000 --- a/doc/user/project/requirements/img/requirements_list_view_v12_10.png +++ /dev/null diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index d9bd02518a4..ae22dbc7e72 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -22,7 +22,7 @@ When a feature is no longer necessary, you can [archive the related requirement] <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [GitLab 12.10 Introduces Requirements Management](https://www.youtube.com/watch?v=uSS7oUNSEoU). - + ## Create a requirement @@ -38,8 +38,6 @@ To create a requirement: You will see the newly created requirement on the top of the list, as the requirements list is sorted by creation date in descending order. - - ## Edit a requirement You can edit a requirement (if you have the necessary privileges) from the requirements @@ -51,16 +49,12 @@ To edit a requirement: 1. Update the title in text input field. 1. Click **Save changes**. - - ## Archive a requirement You can archive an open requirement (if you have the necessary privileges) while you're in the **Open** tab. -From the requirements list page, click **Archive** (**{archive}**). - - +To archive a requirement, click **Archive** (**{archive}**). As soon as a requirement is archived, it no longer appears in the **Open** tab. @@ -68,35 +62,37 @@ As soon as a requirement is archived, it no longer appears in the **Open** tab. You can view the list of archived requirements in the **Archived** tab. - + To reopen an archived requirement, click **Reopen**. As soon as a requirement is reopened, it no longer appears in the **Archived** tab. -## Search for a requirement from the requirements list page +## Search for a requirement -> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212543) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. -You can search for a requirement from the list of requirements using filtered search bar (similar to -that of Issues and Merge Requests) based on following parameters: +You can search for a requirement from the requirements list page based on the following criteria: -- Title -- Author username +- Requirement title +- Author's username -To search, go to the list of requirements and click the field **Search or filter results**. -It will display a dropdown menu, from which you can add an author. You can also enter plain -text to search by epic title or description. When done, press <kbd>Enter</kbd> on your -keyboard to filter the list. +To search for a requirement: -You can also sort requirements list by: +1. In a project, go to **{requirements}** **Requirements > List**. +1. Click the **Search or filter results** field. A dropdown menu appears. +1. Select the requirement author from the dropdown or enter plain text to search by requirement title. +1. Press <kbd>Enter</kbd> on your keyboard to filter the list. + +You can also sort the requirements list by: - Created date - Last updated ## Allow requirements to be satisfied from a CI job -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/215514) ability to specify individual requirements and their statuses in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. GitLab supports [requirements test reports](../../../ci/pipelines/job_artifacts.md#artifactsreportsrequirements-ultimate) now. @@ -132,6 +128,32 @@ the requirement report is checked for the "all passed" record (`{"*":"passed"}`), and on success, it marks all existing open requirements as Satisfied. +#### Specifying individual requirements + +It is possible to specify individual requirements and their statuses. + +If the following requirements exist: + +- `REQ-1` (with IID `1`) +- `REQ-2` (with IID `2`) +- `REQ-3` (with IID `3`) + +It is possible to specify that the first requirement passed, and the second failed. +Valid values are "passed" and "failed". +By omitting a requirement IID (in this case `REQ-3`'s IID `3`), no result is noted. + +```yaml +requirements_confirmation: + when: manual + allow_failure: false + script: + - mkdir tmp + - echo "{\"1\":\"passed\", \"2\":\"failed\"}" > tmp/requirements.json + artifacts: + reports: + requirements: tmp/requirements.json +``` + ### Add the manual job to CI conditionally To configure your CI to include the manual job only when there are some open @@ -140,9 +162,9 @@ requirements, add a rule which checks `CI_HAS_OPEN_REQUIREMENTS` CI variable. ```yaml requirements_confirmation: rules: - - if: "$CI_HAS_OPEN_REQUIREMENTS" == "true" - when: manual - - when: never + - if: "$CI_HAS_OPEN_REQUIREMENTS" == "true" + when: manual + - when: never allow_failure: false script: - mkdir tmp diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index ffb1f6a1407..cb6f8e2221d 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -4,10 +4,11 @@ group: Certify info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Service Desk **(STARTER)** +# Service Desk > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/149) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.1. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/214839) to [GitLab Starter](https://about.gitlab.com/pricing/) in 13.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/215364) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2. ## Overview @@ -61,10 +62,10 @@ users will only see the thread through email. ## Configuring Service Desk NOTE: **Note:** -Service Desk is enabled on GitLab.com. If you're a [Silver subscriber](https://about.gitlab.com/pricing/#gitlab-com), -you can skip step 1 below; you only need to enable it per project. +Service Desk is enabled on GitLab.com. +You can skip step 1 below; you only need to enable it per project. -If you have the correct access and a Premium license, you have the option to set up Service Desk. +If you have project maintainer access you have the option to set up Service Desk. Follow these steps to do so: 1. [Set up incoming email](../../administration/incoming_email.md#set-it-up) for the GitLab instance. @@ -149,7 +150,9 @@ It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (  -For example, suppose you add the following to your configuration: +You can add the following snippets to your configuration. + +Example for installations from source: ```yaml service_desk_email: @@ -167,6 +170,32 @@ service_desk_email: expunge_deleted: true ``` +Example for Omnibus GitLab installations: + +```ruby +gitlab_rails['service_desk_email_enabled'] = true + +gitlab_rails['service_desk_email_address'] = "project_contact+%{key}@gmail.com" + +gitlab_rails['service_desk_email_email'] = "project_support@gmail.com" + +gitlab_rails['service_desk_email_password'] = "[REDACTED]" + +gitlab_rails['service_desk_email_mailbox_name'] = "inbox" + +gitlab_rails['service_desk_email_idle_timeout'] = 60 + +gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" + +gitlab_rails['service_desk_email_host'] = "imap.gmail.com" + +gitlab_rails['service_desk_email_port'] = 993 + +gitlab_rails['service_desk_email_ssl'] = true + +gitlab_rails['service_desk_email_start_tls'] = false +``` + In this case, suppose the `mygroup/myproject` project Service Desk settings has the project name suffix set to `support`, and a user sends an email to `project_contact+mygroup-myproject-support@example.com`. As a result, a new Service Desk issue is created from this email in the `mygroup/myproject` project. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 5a364eb89aa..cb9f0491b44 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -44,6 +44,8 @@ Note the following: ## Version history +### 13.0+ + Starting with GitLab 13.0, GitLab can import bundles that were exported from a different GitLab deployment. This ability is limited to two previous GitLab [minor](../../../policy/maintenance.md#versioning) releases, which is similar to our process for [Security Releases](../../../policy/maintenance.md#security-releases). @@ -61,7 +63,7 @@ Prior to 13.0 this was a defined compatibility table: | Exporting GitLab version | Importing GitLab version | | -------------------------- | -------------------------- | -| 11.7 to 13.0 | 11.7 to 13.0 | +| 11.7 to 12.10 | 11.7 to 12.10 | | 11.1 to 11.6 | 11.1 to 11.6 | | 10.8 to 11.0 | 10.8 to 11.0 | | 10.4 to 10.7 | 10.4 to 10.7 | @@ -95,7 +97,7 @@ The following items will be exported: - Project and wiki repositories - Project uploads -- Project configuration, including services +- Project configuration, excluding integrations - Issues with comments, merge requests with diffs and comments, labels, milestones, snippets, time tracking, and other project entities - Design Management files and data @@ -162,6 +164,15 @@ NOTE: **Note:** The maximum import file size can be set by the Administrator, default is 50MB. As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](../../../api/settings.md#change-application-settings) or the [Admin UI](../../admin_area/settings/account_and_limit_settings.md). +### Project import status + +You can query an import through the [Project import/export API](../../../api/project_import_export.md#import-status). +As described in the API documentation, the query may return an import error or exceptions. + +### Import large projects **(CORE ONLY)** + +If you have a larger project, consider using a Rake task, as described in our [developer documentation](../../../development/import_project.md#importing-via-a-rake-task). + ## Rate limits To help avoid abuse, users are rate limited to: diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 0798c39fff5..7fe6e702d1c 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -20,7 +20,7 @@ Adjust your project's name, description, avatar, [default branch](../repository/ The project description also partially supports [standard Markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. -#### Compliance framework **(ULTIMATE)** +#### Compliance framework **(PREMIUM)** You can select a framework label to identify that your project has certain compliance requirements or needs additional oversight. Available labels include: @@ -68,7 +68,7 @@ Some features depend on others: - If you disable the **Issues** option, GitLab also removes the following features: - **Issue Boards** - - [**Service Desk**](#service-desk-starter) **(STARTER)** + - [**Service Desk**](#service-desk-starter) NOTE: **Note:** When the **Issues** option is disabled, you can still access **Milestones** @@ -223,13 +223,18 @@ To remove a project: 1. In the Remove project section, click the **Remove project** button. 1. Confirm the action when asked to. -This action either: +This action: - Removes a project including all associated resources (issues, merge requests etc). -- Since [GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/32935), on - [GitLab Premium or GitLab.com Silver](https://about.gitlab.com/pricing/) or higher tiers, marks a project for - deletion. The deletion will happen 7 days later by default, but this can be changed in the - [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). +- From [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) on [Premium or Silver](https://about.gitlab.com/pricing/) or higher tiers, +group admins can [configure](../../group/index.md#enabling-delayed-project-removal-premium) projects within a group +to be deleted after a delayed period. +When enabled, actual deletion happens after number of days +specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). + +CAUTION: **Warning:** +The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to +[Immediate deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. #### Restore a project **(PREMIUM)** @@ -269,7 +274,7 @@ Configure Error Tracking to discover and view [Sentry errors within GitLab](../o ### Jaeger tracing **(ULTIMATE)** -Add the URL of a Jaeger server to allow your users to [easily access the Jaeger UI from within GitLab](../operations/tracing.md). +Add the URL of a Jaeger server to allow your users to [easily access the Jaeger UI from within GitLab](../../../operations/tracing.md). ### Status Page diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index 15c3bd5522e..8653bb5001a 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -9,6 +9,7 @@ description: "The static site editor enables users to edit content on static web > - WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214559) in GitLab 13.0. > - Support for adding images through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216640) in GitLab 13.1. > - Markdown front matter hidden on the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216834) in GitLab 13.1. +> - Support for `*.md.erb` files [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223171) in GitLab 13.2. DANGER: **Danger:** In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) @@ -100,5 +101,4 @@ company and a new feature has been added to the company product. ## Limitations -- Currently, the Static Site Editor only works for files ending in `.md`. For example, it will not work for a file `index.html.md.erb` while it works for `index.html.md`. - The Static Site Editor still cannot be quickly added to existing Middleman sites. Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. diff --git a/doc/user/project/status_page/index.md b/doc/user/project/status_page/index.md index ec0a79583d5..1c885ae043f 100644 --- a/doc/user/project/status_page/index.md +++ b/doc/user/project/status_page/index.md @@ -34,10 +34,12 @@ Setting up a Status Page is pretty painless but there are a few things you need To use GitLab Status Page you first need to set up your account details for your cloud provider in the operations settings page. Today, only AWS is supported. -1. Within your AWS account, create an AWS access key. -1. Add the following permissions policies: +#### AWS Setup + +1. Within your AWS acccout, create two new IAM policies. - [Create bucket](https://gitlab.com/gitlab-org/status-page/-/blob/master/deploy/etc/s3_create_policy.json). - [Update bucket contents](https://gitlab.com/gitlab-org/status-page/-/blob/master/deploy/etc/s3_update_bucket_policy.json) (Remember replace `S3_BUCKET_NAME` with your bucket name). +1. Create a new AWS access key with the permissions policies created in the first step. ### Status Page project diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index ce20f2308e6..81f36317b0c 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -24,6 +24,8 @@ file path fragments to start seeing results. ## Syntax highlighting +> Support for `.gitlab.ci.yml` validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218472) in GitLab 13.2. + As expected from an IDE, syntax highlighting for many languages within the Web IDE will make your direct editing even easier. @@ -35,10 +37,21 @@ The Web IDE currently provides: - IntelliSense and validation support (displaying errors and warnings, providing smart completions, formatting, and outlining) for some languages. For example: TypeScript, JavaScript, CSS, LESS, SCSS, JSON, and HTML. +- Validation support for certain JSON and YAML files using schemas based on the + [JSON Schema Store](https://www.schemastore.org/json/). This feature + is only supported for the `.gitlab-ci.yml` file. + + NOTE: **Note:** + Validation support based on schemas is hidden behind + the feature flag `:schema_linting` on self-managed installations. To enable the + feature, you can [turn on the feature flag in Rails console](../../../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags). Because the Web IDE is based on the [Monaco Editor](https://microsoft.github.io/monaco-editor/), you can find a more complete list of supported languages in the -[Monaco languages](https://github.com/Microsoft/monaco-languages) repository. +[Monaco languages](https://github.com/Microsoft/monaco-languages) repository. Under the hood, +Monaco uses the [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for syntax highlighting. + +If you are missing Syntax Highlighting support for any language, we prepared a short guide on how to [add support for a missing language Syntax Highlighting.](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/ide/lib/languages/README.md) NOTE: **Note:** Single file editing is based on the [Ace Editor](https://ace.c9.io). @@ -210,16 +223,20 @@ to work: - The Runner needs to have [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section). + This section requires at least a `session_timeout` value (which defaults to 1800 + seconds) and a `listen_address` value. If `advertise_address` is not defined, `listen_address` is used. - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). **(ULTIMATE ONLY)** If you have the terminal open and the job has finished with its tasks, the terminal will block the job from finishing for the duration configured in -[`[session_server].terminal_max_retention_time`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) +[`[session_server].session_timeout`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) until you close the terminal window. -NOTE: **Note:** Not all executors are -[supported](https://docs.gitlab.com/runner/executors/#compatibility-chart) +NOTE: **Note:** +Not all executors are +[supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). +The [File Sync](#file-syncing-to-web-terminal) feature is supported on Kubernetes runners only. ### Web IDE configuration file @@ -243,6 +260,8 @@ In the code below there is an example of this configuration file: ```yaml terminal: + # This can be any image that has the necessary runtime environment for your project. + image: node:10-alpine before_script: - apt-get update script: sleep 60 diff --git a/doc/user/project/wiki/img/wiki_page_diffs_v13_2.png b/doc/user/project/wiki/img/wiki_page_diffs_v13_2.png Binary files differnew file mode 100644 index 00000000000..261e6e66c97 --- /dev/null +++ b/doc/user/project/wiki/img/wiki_page_diffs_v13_2.png diff --git a/doc/user/project/wiki/img/wiki_page_history.png b/doc/user/project/wiki/img/wiki_page_history.png Binary files differindex 5a1ae295ed2..0cef2345e27 100644 --- a/doc/user/project/wiki/img/wiki_page_history.png +++ b/doc/user/project/wiki/img/wiki_page_history.png diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 82dbeb0ff7e..9044ee0765f 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -134,47 +134,68 @@ The changes of a wiki page over time are recorded in the wiki's Git repository, and you can view them by clicking the **Page history** button. From the history page you can see the revision of the page (Git commit SHA), its -author, the commit message, when it was last updated, and the page markup format. +author, the commit message, and when it was last updated. To see how a previous version of the page looked like, click on a revision -number. +number in the **Page version** column.  +### Viewing the changes between page versions + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15242) in GitLab 13.2. + +Similar to versioned diff file views, you can see the changes made in a given Wiki page version: + +1. Navigate to the Wiki page you're interested in. +1. Click on **Page history** to see all page versions. +1. Click on the commit message in the **Changes** column for the version you're interested in: + +  + ## Wiki activity records -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14902) in GitLab 12.10. -> - It's deployed behind a feature flag, disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14902) in **GitLab 12.10.** +> - Git events were [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216014) in **GitLab 13.0.** > - It's enabled on GitLab.com. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-core-only). **(CORE ONLY)** +> - Git access activity creation is managed by a feature flag. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-in-git-core-only). **(CORE ONLY)** Wiki events (creation, deletion, and updates) are tracked by GitLab and displayed on the [user profile](../../profile/index.md#user-profile), [group](../../group/index.md#view-group-activity), and [project](../index.md#project-activity) activity pages. -### Limitations - -Only edits made in the browser or through the API have their activity recorded. -Edits made and pushed through Git are not currently listed in the activity list. +### Enable or disable Wiki events in Git **(CORE ONLY)** -### Enable or disable Wiki Events **(CORE ONLY)** - -Wiki event activity is under development and not ready for production use. It is +Tracking wiki events through Git is under development and not ready for production use. It is deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session) -can enable it for your instance. You're welcome to test it, but use it at your -own risk. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. To enable it: ```ruby -Feature.enable(:wiki_events) +Feature.enable(:wiki_events_on_git_push) +``` + +To enable for just a particular project: + +```ruby +project = Project.find_by_full_path('your-group/your-project') +Feature.enable(:wiki_events_on_git_push, project) ``` To disable it: ```ruby -Feature.disable(:wiki_events) +Feature.disable(:wiki_events_on_git_push) +``` + +To disable for just a particular project: + +```ruby +project = Project.find_by_full_path('your-group/your-project') +Feature.disable(:wiki_events_on_git_push, project) ``` ## Adding and editing wiki pages locally diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index eaa57395b8f..0613e9b8e34 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -1,11 +1,9 @@ -# Advanced Global Search **(STARTER ONLY)** +# Advanced Global Search **(STARTER)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109) in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4. NOTE: **GitLab.com availability:** -Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. -It will be progressively enabled for all paid groups in Q3 of 2020. -[Follow this epic](https://gitlab.com/groups/gitlab-com/-/epics/649) for the latest updates on the timeline. +Advanced Global Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. Leverage Elasticsearch for faster, more advanced code search across your entire GitLab instance. @@ -22,7 +20,6 @@ now search for code within other teams that can help your own project. GitLab leverages the search capabilities of [Elasticsearch](https://www.elastic.co/elasticsearch/) and enables it when searching in: -- GitLab application - Projects - Repositories - Commits diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index 1ca9649b581..5dc1f8fe779 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -1,13 +1,9 @@ -# Advanced Syntax Search **(STARTER ONLY)** +# Advanced Syntax Search **(STARTER)** -> **Notes:** -> > - Introduced in [GitLab Enterprise Starter](https://about.gitlab.com/pricing/) 9.2 NOTE: **GitLab.com availability:** -Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. -It will be progressively enabled for all paid groups in Q3 of 2020. -[Follow this epic](https://gitlab.com/groups/gitlab-com/-/epics/649) for the latest updates on the timeline. +Advanced Global Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. Use advanced queries for more targeted search results. diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index efa374cf1c3..314fe367ca6 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -78,10 +78,11 @@ These shortcuts are available when viewing issues and merge requests. | <kbd>m</kbd> | Change milestone. | | <kbd>l</kbd> | Change label. | | <kbd>r</kbd> | Start writing a comment. If any text is selected, it will be quoted in the comment. Can't be used to reply within a thread. | -| <kbd>n</kbd> | Move to next unresolved discussion (Merge requests only). | -| <kbd>p</kbd> | Move to previous unresolved discussion (Merge requests only). | -| <kbd>]</kbd> or <kbd>j</kbd> | Move to next file (Merge requests only). | -| <kbd>[</kbd> or <kbd>k</kbd> | Move to previous file (Merge requests only). | +| <kbd>n</kbd> | Move to next unresolved discussion (merge requests only). | +| <kbd>p</kbd> | Move to previous unresolved discussion (merge requests only). | +| <kbd>]</kbd> or <kbd>j</kbd> | Move to next file (merge requests only). | +| <kbd>[</kbd> or <kbd>k</kbd> | Move to previous file (merge requests only). | +| <kbd>b</kbd> | Copy source branch name (merge requests only). | ### Project Files diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 68e5c5ac92c..4d8f47ee3be 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -96,6 +96,14 @@ This allows you to have a local copy of the snippet's repository and make changes as needed. You can commit those changes and push them to the remote master branch. +### Reduce snippets repository size + +Since versioned Snippets are considered as part of the [namespace storage size](../user/admin_area/settings/account_and_limit_settings.md), +it's recommended to keep snippets' repositories as compact as possible. + +For more information about tools to compact repositories, +see the documentation on [reducing repository size](../user/project/repository/reducing_the_repo_size_using_git.md). + ### Limitations - Binary files are not supported. diff --git a/doc/user/todos.md b/doc/user/todos.md index 5d3e3e62652..ef0e75bc197 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -42,8 +42,7 @@ A To Do appears on your To-Do List when: - You are `@mentioned` in a comment on a: - Commit - Design -- A job in the CI pipeline running for your merge request failed, but this - job is not allowed to fail +- The CI/CD pipeline for your merge request failed - An open merge request becomes unmergeable due to conflict, and you are either: - The author - Have set it to automatically merge once the pipeline succeeds |
