From a09983ae35713f5a2bbb100981116d31ce99826e Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Mon, 20 Jul 2020 12:26:25 +0000 Subject: Add latest changes from gitlab-org/gitlab@13-2-stable-ee --- .../admin_area/activating_deactivating_users.md | 2 +- doc/user/admin_area/credentials_inventory.md | 4 +- .../admin_area/img/credentials_inventory_v12_6.png | Bin 52974 -> 0 bytes .../admin_area/img/credentials_inventory_v13_2.png | Bin 0 -> 96526 bytes ..._approval_settings_compliance_project_v13_1.png | Bin 0 -> 36819 bytes .../img/scope_mr_approval_settings_v13_1.png | Bin 0 -> 69238 bytes doc/user/admin_area/license.md | 29 +- doc/user/admin_area/merge_requests_approvals.md | 19 + doc/user/admin_area/monitoring/health_check.md | 8 +- .../settings/account_and_limit_settings.md | 42 +- .../admin_area/settings/continuous_integration.md | 32 +- ...ail_notification_for_unknown_sign_ins_v13_2.png | Bin 0 -> 12539 bytes .../img/import_export_rate_limits_v13_2.png | Bin 0 -> 54802 bytes .../settings/import_export_rate_limits.md | 32 + doc/user/admin_area/settings/index.md | 1 + .../settings/rate_limit_on_issues_creation.md | 7 +- .../admin_area/settings/sign_in_restrictions.md | 27 +- .../settings/visibility_and_access_controls.md | 12 +- doc/user/analytics/value_stream_analytics.md | 6 - .../application_security/configuration/index.md | 16 +- .../img/container_scanning_v13_0.png | Bin 33010 -> 0 bytes .../img/container_scanning_v13_2.png | Bin 0 -> 8658 bytes .../container_scanning/index.md | 25 +- .../application_security/coverage_fuzzing/index.md | 117 ++ .../dast/img/dast_all_v13_0.png | Bin 32346 -> 0 bytes .../dast/img/dast_on_demand_v13_2.png | Bin 0 -> 91775 bytes .../application_security/dast/img/dast_v13_2.png | Bin 0 -> 6763 bytes doc/user/application_security/dast/index.md | 190 ++-- .../dependency_scanning/analyzers.md | 7 + .../img/dependency_scanning_v13_0.png | Bin 44921 -> 0 bytes .../img/dependency_scanning_v13_2.png | Bin 0 -> 10289 bytes .../dependency_scanning/index.md | 13 +- .../img/security_configuration_page_v13_1.png | Bin 199472 -> 0 bytes .../img/security_configuration_page_v13_2.png | Bin 0 -> 51691 bytes doc/user/application_security/index.md | 54 +- doc/user/application_security/sast/analyzers.md | 37 +- .../application_security/sast/img/sast_v13_0.png | Bin 29907 -> 0 bytes .../application_security/sast/img/sast_v13_2.png | Bin 0 -> 7703 bytes doc/user/application_security/sast/index.md | 104 +- .../img/secret-detection-merge-request-ui.png | Bin 100409 -> 0 bytes .../img/secret_detection_v13_2.png | Bin 0 -> 5863 bytes .../application_security/secret_detection/index.md | 21 +- .../group_security_dashboard_export_csv_v13_1.png | Bin 536756 -> 105028 bytes .../img/group_security_dashboard_v13_0.png | Bin 69236 -> 0 bytes .../img/group_security_dashboard_v13_2_noNav.png | Bin 0 -> 53913 bytes ...ance_security_dashboard_with_projects_v13_0.png | Bin 58505 -> 0 bytes ...e_security_dashboard_with_projects_v13_2_sm.png | Bin 0 -> 58332 bytes .../img/pipeline_security_dashboard_v12_6.png | Bin 59799 -> 0 bytes .../img/pipeline_security_dashboard_v13_2.png | Bin 0 -> 73101 bytes .../img/project_security_dashboard_v13_2.png | Bin 0 -> 78549 bytes .../img/standalone_vulnerability_page_v13_1.png | Bin 0 -> 79341 bytes .../img/vulnerability_list_table_v13_1.png | Bin 0 -> 74381 bytes .../security_dashboard/index.md | 129 +-- .../threat_monitoring/index.md | 11 +- .../img/standalone_vulnerability_page_v12_10.png | Bin 26548 -> 0 bytes .../img/standalone_vulnerability_page_v13_1.png | Bin 0 -> 110282 bytes .../application_security/vulnerabilities/index.md | 8 +- doc/user/asciidoc.md | 78 +- doc/user/clusters/applications.md | 333 +++++- doc/user/clusters/crossplane.md | 400 ++++--- doc/user/clusters/management_project.md | 2 +- .../img/compliance_dashboard_v12_10.png | Bin 98355 -> 0 bytes .../img/compliance_dashboard_v13_2.png | Bin 0 -> 84922 bytes doc/user/compliance/compliance_dashboard/index.md | 3 +- .../img/policies_maintainer_add_v13_0.png | Bin 22079 -> 0 bytes .../img/policies_maintainer_add_v13_2.png | Bin 0 -> 13419 bytes .../img/policies_maintainer_edit_v13_0.png | Bin 40712 -> 0 bytes .../img/policies_maintainer_edit_v13_2.png | Bin 0 -> 20327 bytes doc/user/compliance/license_compliance/index.md | 149 +-- doc/user/discussions/index.md | 21 +- doc/user/gitlab_com/index.md | 41 +- doc/user/group/bulk_editing/img/bulk-editing.png | Bin 99844 -> 0 bytes .../group/bulk_editing/img/bulk-editing_v13_2.png | Bin 0 -> 123971 bytes doc/user/group/bulk_editing/index.md | 8 +- doc/user/group/clusters/index.md | 13 +- .../epics/img/epic_activity_sort_order_v13_2.png | Bin 0 -> 20531 bytes doc/user/group/epics/img/epics_list_view_v12.5.png | Bin 116123 -> 0 bytes doc/user/group/epics/img/new_epic_form_v13.2.png | Bin 0 -> 96690 bytes .../group/epics/img/new_epic_from_groups_v13.2.png | Bin 0 -> 78168 bytes doc/user/group/epics/index.md | 25 +- doc/user/group/epics/manage_epics.md | 71 +- doc/user/group/index.md | 64 +- doc/user/group/iterations/index.md | 29 +- doc/user/group/roadmap/img/roadmap_view_v13_0.png | Bin 55012 -> 0 bytes doc/user/group/roadmap/img/roadmap_view_v13_2.png | Bin 0 -> 55061 bytes doc/user/group/roadmap/index.md | 14 +- doc/user/group/saml_sso/group_managed_accounts.md | 116 ++ doc/user/group/saml_sso/index.md | 279 ++--- doc/user/group/saml_sso/scim_setup.md | 10 +- doc/user/group/subgroups/index.md | 18 +- .../img/pagerduty_incidents_integration_13_2.png | Bin 0 -> 34698 bytes doc/user/incident_management/index.md | 43 +- doc/user/index.md | 4 +- .../img/terraform_plan_widget_v13_2.png | Bin 0 -> 33916 bytes doc/user/infrastructure/index.md | 211 ++-- doc/user/markdown.md | 48 +- doc/user/packages/composer_repository/index.md | 4 +- doc/user/packages/conan_repository/index.md | 10 +- .../img/expiration_policy_app_v13_0.png | Bin 61601 -> 0 bytes doc/user/packages/container_registry/index.md | 103 +- .../packages/img/group_packages_list_v13_0.png | Bin 50889 -> 0 bytes doc/user/packages/img/package_detail_v13_0.png | Bin 46047 -> 0 bytes .../packages/img/project_packages_list_v13_0.png | Bin 52752 -> 0 bytes doc/user/packages/index.md | 128 ++- doc/user/packages/maven_repository/index.md | 12 +- doc/user/packages/npm_registry/index.md | 9 +- doc/user/packages/nuget_repository/index.md | 2 +- doc/user/packages/pypi_repository/index.md | 2 +- doc/user/permissions.md | 18 +- doc/user/profile/account/delete_account.md | 3 +- .../profile/account/two_factor_authentication.md | 6 +- doc/user/profile/index.md | 2 +- doc/user/profile/notifications.md | 2 +- doc/user/profile/personal_access_tokens.md | 5 + doc/user/profile/preferences.md | 5 +- doc/user/profile/unknown_sign_in_notification.md | 12 +- doc/user/project/bulk_editing.md | 6 +- doc/user/project/clusters/add_remove_clusters.md | 80 +- doc/user/project/clusters/img/rbac.png | Bin 15960 -> 0 bytes doc/user/project/clusters/img/rbac_v13_1.png | Bin 0 -> 10680 bytes doc/user/project/clusters/index.md | 281 ++--- doc/user/project/clusters/kubernetes_pod_logs.md | 4 +- .../project/clusters/runbooks/img/helm-install.png | Bin 71705 -> 0 bytes doc/user/project/clusters/runbooks/index.md | 10 +- doc/user/project/clusters/securing.md | 154 +++ doc/user/project/clusters/serverless/aws.md | 12 +- doc/user/project/clusters/serverless/index.md | 50 +- doc/user/project/code_intelligence.md | 54 + doc/user/project/code_owners.md | 163 ++- doc/user/project/deploy_tokens/index.md | 27 +- doc/user/project/description_templates.md | 2 +- doc/user/project/highlighting.md | 10 +- doc/user/project/img/bulk-editing.png | Bin 197667 -> 0 bytes doc/user/project/img/bulk-editing_v13_2.png | Bin 0 -> 132734 bytes doc/user/project/img/code_intelligence_v13_1.png | Bin 0 -> 83690 bytes .../project/img/sectional_code_owners_v13.2.png | Bin 0 -> 106361 bytes doc/user/project/import/gemnasium.md | 2 +- .../jira/import_issues_from_jira_button_v12_10.png | Bin 8504 -> 8422 bytes .../jira/import_issues_from_jira_form_v12_10.png | Bin 116641 -> 56306 bytes .../jira/import_issues_from_jira_form_v13_2.png | Bin 0 -> 108152 bytes .../import_issues_from_jira_projects_v12_10.png | Bin 521845 -> 0 bytes doc/user/project/import/jira.md | 33 +- doc/user/project/index.md | 93 +- doc/user/project/insights/index.md | 21 + doc/user/project/integrations/bugzilla.md | 1 - .../project/integrations/custom_issue_tracker.md | 2 - doc/user/project/integrations/generic_alerts.md | 63 +- doc/user/project/integrations/github.md | 2 +- .../actions_menu_create_new_dashboard_v13_2.png | Bin 0 -> 11479 bytes .../img/heatmap_chart_too_much_data_v_13_2.png | Bin 0 -> 7310 bytes .../img/jira/open_jira_issues_list_v13.2.png | Bin 0 -> 90251 bytes .../integrations/img/jira_service_page_v12_2.png | Bin 57327 -> 0 bytes .../img/metrics_settings_button_v13_2.png | Bin 0 -> 1901 bytes .../img/prometheus_manual_configuration_v13_2.png | Bin 0 -> 15651 bytes .../img/prometheus_service_configuration.png | Bin 5022 -> 0 bytes doc/user/project/integrations/jira.md | 95 +- .../integrations/jira_cloud_configuration.md | 2 +- .../integrations/jira_server_configuration.md | 2 +- doc/user/project/integrations/overview.md | 21 +- doc/user/project/integrations/prometheus.md | 1124 +------------------- .../prometheus_library/nginx_ingress.md | 3 +- .../prometheus_library/nginx_ingress_vts.md | 3 +- doc/user/project/integrations/prometheus_units.md | 174 +-- doc/user/project/integrations/redmine.md | 1 - .../project/integrations/services_templates.md | 27 +- doc/user/project/integrations/slack.md | 60 +- doc/user/project/integrations/youtrack.md | 1 - doc/user/project/issue_board.md | 24 - doc/user/project/issues/crosslinking_issues.md | 3 +- doc/user/project/issues/csv_import.md | 3 +- doc/user/project/issues/design_management.md | 71 +- .../img/design_drag_and_drop_uploads_v13_2.png | Bin 0 -> 1260905 bytes .../issues/img/design_management_upload_v13.2.png | Bin 0 -> 62146 bytes .../project/issues/img/design_management_v13_2.png | Bin 0 -> 1017975 bytes doc/user/project/issues/index.md | 2 - doc/user/project/issues/issue_data_and_actions.md | 4 +- doc/user/project/issues/managing_issues.md | 5 +- doc/user/project/labels.md | 17 +- doc/user/project/members/index.md | 27 + .../merge_requests/accessibility_testing.md | 8 +- .../merge_requests/browser_performance_testing.md | 164 +-- doc/user/project/merge_requests/code_quality.md | 56 +- .../project/merge_requests/fail_fast_testing.md | 87 ++ doc/user/project/merge_requests/getting_started.md | 2 +- .../img/browser_performance_testing.png | Bin 52100 -> 95312 bytes .../img/draft_blocked_merge_button_v13_2.png | Bin 0 -> 32770 bytes .../merge_requests/img/file_by_file_v13_2.png | Bin 0 -> 81874 bytes .../img/load_performance_testing.png | Bin 0 -> 60196 bytes ...when_pipeline_succeeds_only_if_succeeds_msg.png | Bin 5237 -> 0 bytes .../img/wip_blocked_accept_button.png | Bin 4970 -> 0 bytes doc/user/project/merge_requests/index.md | 2 +- .../merge_requests/load_performance_testing.md | 197 ++++ .../merge_requests/merge_request_dependencies.md | 6 +- .../merge_requests/merge_when_pipeline_succeeds.md | 87 +- .../reviewing_and_managing_merge_requests.md | 37 + .../project/merge_requests/squash_and_merge.md | 54 + .../merge_requests/test_coverage_visualization.md | 3 + .../testing_and_reports_in_merge_requests.md | 8 +- .../work_in_progress_merge_requests.md | 34 +- doc/user/project/milestones/index.md | 52 +- doc/user/project/operations/alert_management.md | 130 ++- doc/user/project/operations/dashboard_settings.md | 5 + doc/user/project/operations/feature_flags.md | 260 +---- .../operations/img/alert_detail_metrics_v13_2.png | Bin 0 -> 27616 bytes .../operations/img/alert_list_search_v13_1.png | Bin 0 -> 12166 bytes .../operations/img/alert_list_sort_v13_1.png | Bin 0 -> 13919 bytes .../project/operations/img/alert_list_v13_1.png | Bin 0 -> 38265 bytes .../operations/img/alert_management_1_v13_0.png | Bin 19152 -> 0 bytes .../operations/img/alert_management_1_v13_1.png | Bin 40053 -> 0 bytes doc/user/project/operations/index.md | 16 +- doc/user/project/operations/tracing.md | 39 +- .../custom_domains_ssl_tls_certification/index.md | 2 +- .../pages/getting_started/fork_sample_project.md | 55 +- .../getting_started/new_or_existing_website.md | 48 +- .../getting_started/pages_bundled_template.md | 33 +- .../pages/getting_started/pages_ci_cd_template.md | 49 + .../getting_started/pages_forked_sample_project.md | 56 + .../pages/getting_started/pages_from_scratch.md | 402 +++++++ .../getting_started/pages_new_project_template.md | 34 + .../project/pages/getting_started_part_four.md | 401 +------ doc/user/project/pages/index.md | 10 +- doc/user/project/pages/introduction.md | 20 +- doc/user/project/protected_tags.md | 2 +- doc/user/project/quick_actions.md | 15 +- .../img/custom_notifications_dropdown_v12_5.png | Bin 14983 -> 0 bytes .../img/custom_notifications_new_release_v12_5.png | Bin 20811 -> 0 bytes .../releases/img/edit_release_page_v13_0.png | Bin 285708 -> 0 bytes .../img/milestone_list_with_releases_v12_5.png | Bin 45454 -> 105666 bytes .../releases/img/milestone_with_releases_v12_5.png | Bin 67529 -> 0 bytes doc/user/project/releases/img/new_tag_12_5.png | Bin 41120 -> 0 bytes .../releases/img/release_edit_button_v12_6.png | Bin 25953 -> 0 bytes .../img/release_milestone_dropdown_v13_0.png | Bin 138986 -> 0 bytes .../releases/img/release_with_milestone_v12_9.png | Bin 27783 -> 57689 bytes .../project/releases/img/releases_count_v13_2.png | Bin 0 -> 27254 bytes doc/user/project/releases/img/releases_v12_9.png | Bin 51974 -> 0 bytes doc/user/project/releases/img/tags_12_5.png | Bin 31541 -> 0 bytes doc/user/project/releases/index.md | 478 +++++---- doc/user/project/repository/branches/index.md | 50 +- doc/user/project/repository/index.md | 3 +- .../repository/reducing_the_repo_size_using_git.md | 33 +- .../project/repository/repository_mirroring.md | 8 +- .../repository/x509_signed_commits/index.md | 6 +- .../img/requirement_archive_view_v12_10.png | Bin 112233 -> 0 bytes .../img/requirement_create_view_v12_10.png | Bin 124402 -> 0 bytes .../img/requirement_edit_view_v12_10.png | Bin 118066 -> 0 bytes .../img/requirements_archived_list_view_v12_10.png | Bin 68623 -> 0 bytes .../img/requirements_archived_list_view_v13_1.png | Bin 0 -> 47662 bytes .../requirements/img/requirements_list_v13_1.png | Bin 0 -> 113403 bytes .../img/requirements_list_view_v12_10.png | Bin 117250 -> 0 bytes doc/user/project/requirements/index.md | 70 +- doc/user/project/service_desk.md | 39 +- doc/user/project/settings/import_export.md | 15 +- doc/user/project/settings/index.md | 21 +- doc/user/project/static_site_editor/index.md | 2 +- doc/user/project/status_page/index.md | 6 +- doc/user/project/web_ide/index.md | 27 +- .../project/wiki/img/wiki_page_diffs_v13_2.png | Bin 0 -> 53576 bytes doc/user/project/wiki/img/wiki_page_history.png | Bin 12101 -> 13456 bytes doc/user/project/wiki/index.md | 55 +- doc/user/search/advanced_global_search.md | 7 +- doc/user/search/advanced_search_syntax.md | 8 +- doc/user/shortcuts.md | 9 +- doc/user/snippets.md | 8 + doc/user/todos.md | 3 +- 264 files changed, 5014 insertions(+), 4203 deletions(-) delete mode 100644 doc/user/admin_area/img/credentials_inventory_v12_6.png create mode 100644 doc/user/admin_area/img/credentials_inventory_v13_2.png create mode 100644 doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png create mode 100644 doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png create mode 100644 doc/user/admin_area/settings/img/email_notification_for_unknown_sign_ins_v13_2.png create mode 100644 doc/user/admin_area/settings/img/import_export_rate_limits_v13_2.png create mode 100644 doc/user/admin_area/settings/import_export_rate_limits.md delete mode 100644 doc/user/application_security/container_scanning/img/container_scanning_v13_0.png create mode 100644 doc/user/application_security/container_scanning/img/container_scanning_v13_2.png create mode 100644 doc/user/application_security/coverage_fuzzing/index.md delete mode 100644 doc/user/application_security/dast/img/dast_all_v13_0.png create mode 100644 doc/user/application_security/dast/img/dast_on_demand_v13_2.png create mode 100644 doc/user/application_security/dast/img/dast_v13_2.png delete mode 100644 doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png create mode 100644 doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png delete mode 100644 doc/user/application_security/img/security_configuration_page_v13_1.png create mode 100644 doc/user/application_security/img/security_configuration_page_v13_2.png delete mode 100644 doc/user/application_security/sast/img/sast_v13_0.png create mode 100644 doc/user/application_security/sast/img/sast_v13_2.png delete mode 100644 doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png create mode 100644 doc/user/application_security/secret_detection/img/secret_detection_v13_2.png delete mode 100644 doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png create mode 100644 doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png delete mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png create mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png delete mode 100644 doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png create mode 100644 doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png create mode 100644 doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png create mode 100644 doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png create mode 100644 doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png delete mode 100644 doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v12_10.png create mode 100644 doc/user/application_security/vulnerabilities/img/standalone_vulnerability_page_v13_1.png delete mode 100644 doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_10.png create mode 100644 doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_2.png delete mode 100644 doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png create mode 100644 doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_2.png delete mode 100644 doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png create mode 100644 doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_2.png delete mode 100644 doc/user/group/bulk_editing/img/bulk-editing.png create mode 100644 doc/user/group/bulk_editing/img/bulk-editing_v13_2.png create mode 100644 doc/user/group/epics/img/epic_activity_sort_order_v13_2.png delete mode 100644 doc/user/group/epics/img/epics_list_view_v12.5.png create mode 100644 doc/user/group/epics/img/new_epic_form_v13.2.png create mode 100644 doc/user/group/epics/img/new_epic_from_groups_v13.2.png delete mode 100644 doc/user/group/roadmap/img/roadmap_view_v13_0.png create mode 100644 doc/user/group/roadmap/img/roadmap_view_v13_2.png create mode 100644 doc/user/group/saml_sso/group_managed_accounts.md create mode 100644 doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png create mode 100644 doc/user/infrastructure/img/terraform_plan_widget_v13_2.png delete mode 100644 doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png delete mode 100644 doc/user/packages/img/group_packages_list_v13_0.png delete mode 100644 doc/user/packages/img/package_detail_v13_0.png delete mode 100644 doc/user/packages/img/project_packages_list_v13_0.png delete mode 100644 doc/user/project/clusters/img/rbac.png create mode 100644 doc/user/project/clusters/img/rbac_v13_1.png delete mode 100644 doc/user/project/clusters/runbooks/img/helm-install.png create mode 100644 doc/user/project/clusters/securing.md create mode 100644 doc/user/project/code_intelligence.md delete mode 100644 doc/user/project/img/bulk-editing.png create mode 100644 doc/user/project/img/bulk-editing_v13_2.png create mode 100644 doc/user/project/img/code_intelligence_v13_1.png create mode 100644 doc/user/project/img/sectional_code_owners_v13.2.png create mode 100644 doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png delete mode 100644 doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png create mode 100644 doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png create mode 100644 doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png create mode 100644 doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png delete mode 100644 doc/user/project/integrations/img/jira_service_page_v12_2.png create mode 100644 doc/user/project/integrations/img/metrics_settings_button_v13_2.png create mode 100644 doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png delete mode 100644 doc/user/project/integrations/img/prometheus_service_configuration.png create mode 100644 doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png create mode 100644 doc/user/project/issues/img/design_management_upload_v13.2.png create mode 100644 doc/user/project/issues/img/design_management_v13_2.png create mode 100644 doc/user/project/merge_requests/fail_fast_testing.md create mode 100644 doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png create mode 100644 doc/user/project/merge_requests/img/file_by_file_v13_2.png create mode 100644 doc/user/project/merge_requests/img/load_performance_testing.png delete mode 100644 doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png delete mode 100644 doc/user/project/merge_requests/img/wip_blocked_accept_button.png create mode 100644 doc/user/project/merge_requests/load_performance_testing.md create mode 100644 doc/user/project/operations/img/alert_detail_metrics_v13_2.png create mode 100644 doc/user/project/operations/img/alert_list_search_v13_1.png create mode 100644 doc/user/project/operations/img/alert_list_sort_v13_1.png create mode 100644 doc/user/project/operations/img/alert_list_v13_1.png delete mode 100644 doc/user/project/operations/img/alert_management_1_v13_0.png delete mode 100644 doc/user/project/operations/img/alert_management_1_v13_1.png create mode 100644 doc/user/project/pages/getting_started/pages_ci_cd_template.md create mode 100644 doc/user/project/pages/getting_started/pages_forked_sample_project.md create mode 100644 doc/user/project/pages/getting_started/pages_from_scratch.md create mode 100644 doc/user/project/pages/getting_started/pages_new_project_template.md delete mode 100644 doc/user/project/releases/img/custom_notifications_dropdown_v12_5.png delete mode 100644 doc/user/project/releases/img/custom_notifications_new_release_v12_5.png delete mode 100644 doc/user/project/releases/img/edit_release_page_v13_0.png delete mode 100644 doc/user/project/releases/img/milestone_with_releases_v12_5.png delete mode 100644 doc/user/project/releases/img/new_tag_12_5.png delete mode 100644 doc/user/project/releases/img/release_edit_button_v12_6.png delete mode 100644 doc/user/project/releases/img/release_milestone_dropdown_v13_0.png create mode 100644 doc/user/project/releases/img/releases_count_v13_2.png delete mode 100644 doc/user/project/releases/img/releases_v12_9.png delete mode 100644 doc/user/project/releases/img/tags_12_5.png delete mode 100644 doc/user/project/requirements/img/requirement_archive_view_v12_10.png delete mode 100644 doc/user/project/requirements/img/requirement_create_view_v12_10.png delete mode 100644 doc/user/project/requirements/img/requirement_edit_view_v12_10.png delete mode 100644 doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png create mode 100644 doc/user/project/requirements/img/requirements_archived_list_view_v13_1.png create mode 100644 doc/user/project/requirements/img/requirements_list_v13_1.png delete mode 100644 doc/user/project/requirements/img/requirements_list_view_v12_10.png create mode 100644 doc/user/project/wiki/img/wiki_page_diffs_v13_2.png (limited to 'doc/user') diff --git a/doc/user/admin_area/activating_deactivating_users.md b/doc/user/admin_area/activating_deactivating_users.md index 06a26737495..448c65038c2 100644 --- a/doc/user/admin_area/activating_deactivating_users.md +++ b/doc/user/admin_area/activating_deactivating_users.md @@ -39,7 +39,7 @@ A user can be deactivated from the Admin Area. To do this: Please note that for the deactivation option to be visible to an admin, the user: - Must be currently active. -- Must not have any signins or activity in the last 180 days. +- Must not have signed in, or have any activity, in the last 180 days. Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 5eecfbb73c6..9259c93cfa3 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -18,9 +18,11 @@ Using Credentials inventory, GitLab administrators can see all the personal acce - Who they belong to. - Their access scope. - Their usage pattern. +- When they expire. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. +- When they were revoked. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. To access the Credentials inventory, navigate to **Admin Area > Credentials**. The following is an example of the Credentials inventory page: -![Credentials inventory page](img/credentials_inventory_v12_6.png) +![Credentials inventory page](img/credentials_inventory_v13_2.png) diff --git a/doc/user/admin_area/img/credentials_inventory_v12_6.png b/doc/user/admin_area/img/credentials_inventory_v12_6.png deleted file mode 100644 index 5c16781cb2d..00000000000 Binary files a/doc/user/admin_area/img/credentials_inventory_v12_6.png and /dev/null differ diff --git a/doc/user/admin_area/img/credentials_inventory_v13_2.png b/doc/user/admin_area/img/credentials_inventory_v13_2.png new file mode 100644 index 00000000000..5b56422a0a3 Binary files /dev/null and b/doc/user/admin_area/img/credentials_inventory_v13_2.png differ diff --git a/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png b/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png new file mode 100644 index 00000000000..04d01f2662f Binary files /dev/null and b/doc/user/admin_area/img/mr_approval_settings_compliance_project_v13_1.png differ diff --git a/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png b/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png new file mode 100644 index 00000000000..c6ca2bac83c Binary files /dev/null and b/doc/user/admin_area/img/scope_mr_approval_settings_v13_1.png differ diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 1ffaf4e0678..c5e29612596 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Activate all GitLab Enterprise Edition functionality with a license **(STARTER ONLY)** +# Activate GitLab EE with a license **(STARTER ONLY)** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload a license. Once you've received your license from GitLab Inc., you can upload it @@ -107,14 +107,23 @@ expired license(s). It's possible to upload and view more than one license, but only the latest license will be used as the active license. - +If you originally installed Community Edition rather than Enterprise Edition you will need to +[upgrade to Enterprise Edition](../../update/README.md#community-to-enterprise-edition) +before uploading your license. + +GitLab.com users cannot upload and use a self-managed license. If you +wish to use paid features on GitLab.com, a separate subscription may be +[purchased](../../subscriptions/index.md#subscribe-to-gitlabcom). + +### Users exceed license limit upon renewal + +If you've added new users to your GitLab instance prior to renewal you may need to +purchase additional seats to cover those users. If this is the case and a license +without enough users is uploaded a message is displayed prompting you to purchase +additional users. More information on how to determine the required number of users +and how to add additional seats can be found in the +[licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). diff --git a/doc/user/admin_area/merge_requests_approvals.md b/doc/user/admin_area/merge_requests_approvals.md index 153ccfc128a..6d9d634ce14 100644 --- a/doc/user/admin_area/merge_requests_approvals.md +++ b/doc/user/admin_area/merge_requests_approvals.md @@ -34,3 +34,22 @@ Merge request approval rules that can be set at an instance level are: - **Prevent users from modifying merge request approvers list**. Prevents project maintainers from allowing users to modify the approvers list in project settings or in individual merge requests. + +## Scope rules to compliance-labeled projects + +> Introduced in [GitLab Premium](https://gitlab.com/groups/gitlab-org/-/epics/3432) 13.2. + +Merge request approval rules can be further scoped to specific compliance frameworks. + +When the compliance framework label is selected and the project is assigned the compliance +label, the instance-level MR approval settings will take effect and the +[project-level settings](../project/merge_requests/merge_request_approvals.md#adding--editing-a-default-approval-rule) +is locked for modification. + +When the compliance framework label is not selected or the project is not assigned the +compliance label, the project-level MR approval settings will take effect and the users with +Maintainer role and above can modify these. + +| Instance-level | Project-level | +| -------------- | ------------- | +| ![Scope MR approval settings to compliance frameworks](img/scope_mr_approval_settings_v13_1.png) | ![MR approval settings on compliance projects](img/mr_approval_settings_compliance_project_v13_1.png) | diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 91e29118e3e..329b6ff5bb0 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -137,8 +137,8 @@ This check is being exempt from Rack Attack. ## Access token (Deprecated) -> NOTE: **Note:** -> Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). +NOTE: **Note:** +Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the **Admin Area > Monitoring > Health check** @@ -152,6 +152,10 @@ The access token can be passed as a URL parameter: https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` +NOTE: **Note:** +In case the database or Redis service are unaccessible, the probe endpoints response is not guaranteed to be correct. +You should switch to [IP whitelist](#ip-whitelist) from deprecated access token to avoid it. + - ## Optimizing DAST By default, DAST will download all artifacts defined by previous jobs in the pipeline. If @@ -734,3 +756,15 @@ variables: Here, DAST is being allocated 3072 MB. Change the number after `-Xmx` to the required memory amount. + + diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 474f9339d0b..ca2b212ffc3 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -1,3 +1,10 @@ +--- +type: reference, howto +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Dependency Scanning Analyzers **(ULTIMATE)** Dependency Scanning relies on underlying third party tools that are wrapped into diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png deleted file mode 100644 index 9f3990df957..00000000000 Binary files a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png and /dev/null differ diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png new file mode 100644 index 00000000000..28c4eb85b7c Binary files /dev/null and b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_2.png differ diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 84ec0ec976d..57b4fae3230 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -27,7 +27,7 @@ GitLab checks the Dependency Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. -![Dependency Scanning Widget](img/dependency_scanning_v13_0.png) +![Dependency Scanning Widget](img/dependency_scanning_v13_2.png) The results are sorted by the severity of the vulnerability: @@ -61,7 +61,7 @@ The following languages and dependency managers are supported: | Language (package managers) | Supported files | Scan tool(s) | |----------------------------- | --------------- | ------------ | | Java ([Gradle](https://gradle.org/), [Maven](https://maven.apache.org/)) | `build.gradle`, `build.gradle.kts`, `pom.xml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js) | +| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | | Go ([Golang](https://golang.org/)) | `go.sum` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | PHP ([Composer](https://getcomposer.org/)) | `composer.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Python ([setuptools](https://setuptools.readthedocs.io/en/latest/), [pip](https://pip.pypa.io/en/stable/), [Pipenv](https://pipenv.pypa.io/en/latest/)) | `setup.py`, `requirements.txt`, `requirements.pip`, `requires.txt`, `Pipfile` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | @@ -72,7 +72,7 @@ Plans are underway for supporting the following languages, dependency managers, | Language (package managers) | Supported files | Scan tool(s) | Issue | |----------------------------- | --------------- | ------------ | ----- | -| Python ([Poetry](https://poetry.eustace.io/)) | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/issues/7006) | +| Python ([Poetry](https://python-poetry.org/)) | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | | Python ([Pipenv](https://pipenv.pypa.io/en/latest/)) | `Pipfile.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#11756](https://gitlab.com/gitlab-org/gitlab/-/issues/11756) | ## Contribute your scanner @@ -151,11 +151,11 @@ The following variables allow configuration of global dependency scanning settin | Environment variable | Description | | --------------------------------------- |------------ | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `DS_ANALYZER_IMAGE_PREFIX` | **DEPRECATED:** Use `SECURE_ANALYZERS_PREFIX` instead. | | `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | | `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"` | +| `SECURE_LOG_LEVEL` | Default log level is `info`, you can set it to any of the following strings: `fatal`, `error`, `warn`, `info`, `debug`. | #### Configuring Docker-in-Docker orchestrator @@ -186,6 +186,7 @@ The following variables are used for configuring specific analyzers (used for a | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | | `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12296) in GitLab 12.1)| +| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`. Maven and Gradle will use the Java version specified by this value. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repos). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | @@ -428,14 +429,14 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set Dependency Scanning CI job variables to use local Dependency Scanning analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must replace -`DS_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: +`SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml variables: - DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers" + SECURE_ANALYZERS_PREFIX: "docker-registry.example.com/analyzers" GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" GIT_SSL_NO_VERIFY: "true" ``` diff --git a/doc/user/application_security/img/security_configuration_page_v13_1.png b/doc/user/application_security/img/security_configuration_page_v13_1.png deleted file mode 100644 index 176c64a9e87..00000000000 Binary files a/doc/user/application_security/img/security_configuration_page_v13_1.png and /dev/null differ diff --git a/doc/user/application_security/img/security_configuration_page_v13_2.png b/doc/user/application_security/img/security_configuration_page_v13_2.png new file mode 100644 index 00000000000..016328948cc Binary files /dev/null and b/doc/user/application_security/img/security_configuration_page_v13_2.png differ diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 49580f494a2..3aca4c59423 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -116,6 +116,44 @@ information with several options: ![Interacting with security reports](img/interacting_with_vulnerability_v13_0.png) +### View details of a DAST vulnerability + +Vulnerabilities detected by DAST occur in the live web application. Rectification of these types of +vulnerabilities requires specific information. DAST provides the information required to +investigate and rectify the underlying cause. + +To view details of DAST vulnerabilities: + +1. To see all vulnerabilities detected: + + - In a project, go to the project's **{shield}** **Security & Compliance** page. + - Only in a merge request, go the merge request's **Security** tab. + +1. Click on the vulnerability's description. The following details are provided: + + | Field | Description | +|:-----------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Description | Description of the vulnerability. | +| Project | Namespace and project in which the vulnerability was detected. | +| Method | HTTP method used to detect the vulnerability. | +| URL | URL at which the vulnerability was detected. | +| Request Headers | Headers of the request. | +| Response Status | Response status received from the application. | +| Response Headers | Headers of the response received from the application. | +| Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. | +| Identifiers | Identifiers of the vulnerability. | +| Severity | Severity of the vulnerability. | +| Scanner Type | Type of vulnerability report. | +| Links | Links to further details of the detected vulnerability. | +| Solution | Details of a recommended solution to the vulnerability (optional). | + +#### Hide sensitive information in headers + +HTTP request and response headers may contain sensitive information, including cookies and +authorization credentials. By default, content of specific headers are masked in DAST vulnerability +reports. You can specify the list of all headers to be masked. For details, see +[Hide sensitive information](dast/index.md#hide-sensitive-information). + ### Dismissing a vulnerability To dismiss a vulnerability, you must set its status to Dismissed. Follow these steps to do so: @@ -258,14 +296,16 @@ An approval is optional when a security report: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -To enable License Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) -must be created with the case-sensitive name `License-Check`. This approval group must be set -with the number of approvals required greater than zero. +`License-Check` is an approval rule you can enable to allow an individual or group to approve a +merge request that contains a `denied` license. + +You can enable `License-Check` one of two ways: -Once this group is added to your project, the approval rule is enabled for all Merge Requests. To -configure how this rule behaves, you can choose which licenses to `allow` or `deny` in the -[project policies for License Compliance](../compliance/license_compliance/index.md#project-policies-for-license-compliance) -section. +- Create a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) + with the case-sensitive name `License-Check`. +- Create an approval group in the [project policies section for License Compliance](../compliance/license_compliance/index.md#policies). + You must set this approval group's number of approvals required to greater than zero. Once you + enable this group in your project, the approval rule is enabled for all merge requests. Any code changes cause the approvals required to reset. diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 0aa20bf4373..214044ad783 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -32,7 +32,6 @@ SAST supports the following official analyzers: - [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)) - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) -- [`tslint`](https://gitlab.com/gitlab-org/security-products/analyzers/tslint) (TSLint (TypeScript)) The analyzers are published as Docker images that SAST will use to launch dedicated containers for each analysis. @@ -145,24 +144,24 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Analyzers Data -| Property \ Tool | Apex | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | TSLint Security | -| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | :-------------: | -| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | ✓ | -| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | -| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | +| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | +| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | - ✓ => we have that data - ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content diff --git a/doc/user/application_security/sast/img/sast_v13_0.png b/doc/user/application_security/sast/img/sast_v13_0.png deleted file mode 100644 index b4aea6ea466..00000000000 Binary files a/doc/user/application_security/sast/img/sast_v13_0.png and /dev/null differ diff --git a/doc/user/application_security/sast/img/sast_v13_2.png b/doc/user/application_security/sast/img/sast_v13_2.png new file mode 100644 index 00000000000..5697ed9beb0 Binary files /dev/null and b/doc/user/application_security/sast/img/sast_v13_2.png differ diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a5497e3d38c..70d4b513cf9 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -9,9 +9,9 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3775) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.3. -NOTE: **4 of the top 6 attacks were application based.** -Download our whitepaper, -["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +NOTE: **Note:** +The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +explains how **4 of the top 6 attacks were application based**. Download it to learn how to protect your organization. ## Overview @@ -28,7 +28,7 @@ You can take advantage of SAST by doing one of the following: GitLab checks the SAST report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. -![SAST Widget](img/sast_v13_0.png) +![SAST Widget](img/sast_v13_2.png) The results are sorted by the priority of the vulnerability: @@ -58,7 +58,8 @@ If you're using the shared Runners on GitLab.com, this is enabled by default. Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker). -CAUTION: **Caution:** Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. +CAUTION: **Caution:** +Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** If you use your own Runners, make sure the Docker version installed @@ -70,31 +71,54 @@ The following table shows which languages, package managers and frameworks are s | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| -| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | -| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | -| Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | -| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | -| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | -| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | -| Go | [Gosec](https://github.com/securego/gosec) | 10.7 | -| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | -| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | +| Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | +| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | +| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | +| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | +| Go | [Gosec](https://github.com/securego/gosec) | 10.7 | +| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | +| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | | Java ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT) | -| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | +| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8, moved to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2 | | Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | | Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | | React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 | -| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | +| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3, moved to [GitLab Core](https://about.gitlab.com/pricing/) in 13.1 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | -| TypeScript | [`tslint-config-security`](https://github.com/webschik/tslint-config-security/) | 11.9 | +| TypeScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.9, merged with ESLint in 13.2 | NOTE: **Note:** The Java analyzers can also be used for variants like the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html), [Grails](https://grails.org/) and the [Maven wrapper](https://github.com/takari/maven-wrapper). +### Making SAST analyzers available to all GitLab tiers + +All open source (OSS) analyzers are in the process of being reviewed and potentially moved to the GitLab Core tier. Progress can be +tracked in the corresponding +[epic](https://gitlab.com/groups/gitlab-org/-/epics/2098). + +Please note that support for [Docker-in-Docker](#enabling-docker-in-docker) +will not be extended to the GitLab Core tier. + +#### Summary of features per tier + +Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), +as shown in the following table: + +| Capability | In Core | In Ultimate | +|:--------------------------------------------------------------------------|:--------------------|:-------------------| +| [Configure SAST Scanners](#configuration) | **{check-circle}** | **{check-circle}** | +| [Customize SAST Settings](#customizing-the-sast-settings) | **{check-circle}** | **{check-circle}** | +| View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | +| [Presentation of JSON Report in Merge Request](#overview) | **{dotted-circle}** | **{check-circle}** | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) | **{dotted-circle}** | **{check-circle}** | +| [Access to Security Dashboard](#security-dashboard) | **{dotted-circle}** | **{check-circle}** | + ## Contribute your scanner The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate other security scanners into GitLab. @@ -222,7 +246,7 @@ a `before_script` execution to prepare your scan job. To pass your project's dependencies as artifacts, the dependencies must be included in the project's working directory and specified using the `artifacts:path` configuration. -If all dependencies are present, the `-compile=false` flag can be provided to the +If all dependencies are present, the `COMPILE=false` variable can be provided to the analyzer and compilation will be skipped: ```yaml @@ -247,10 +271,9 @@ build: spotbugs-sast: dependencies: - build - script: - - /analyzer run -compile=false variables: MAVEN_REPO_PATH: ./.m2/repository + COMPILE: false artifacts: reports: sast: gl-sast-report.json @@ -266,6 +289,16 @@ See [Analyzer settings](#analyzer-settings) for the complete list of available o SAST can be [configured](#customizing-the-sast-settings) using environment variables. +#### Logging Level + +You can control the verbosity of logs by setting the `SECURE_LOG_LEVEL` env var. The default is set to `info`, you can set it to any of the following levels: + +- `fatal` +- `error` +- `warn` +- `info` +- `debug` + #### Custom Certificate Authority To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle @@ -278,7 +311,6 @@ The following are Docker image-related variables. | Environment variable | Description | |------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `SAST_ANALYZER_IMAGE_PREFIX` | **DEPRECATED**: Use `SECURE_ANALYZERS_PREFIX` instead. | | `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | @@ -287,17 +319,18 @@ The following are Docker image-related variables. Some analyzers make it possible to filter out vulnerabilities under a given threshold. -| Environment variable | Default value | Description | -|-------------------------|---------------|-------------| +| Environment variable | Default value | Description | +|-------------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories will also match patterns. | -| `SAST_BANDIT_EXCLUDED_PATHS` | - | comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*'` | -| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | -| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | -| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | -| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | -| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | -| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | -| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +| `SAST_BANDIT_EXCLUDED_PATHS` | | Comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*, */venv/*'` | +| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | +| `SAST_DISABLE_BABEL` | `false` | Disable Babel processing for the NodeJsScan scanner. Set to `true` to disable Babel processing. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33065) in GitLab 13.2. | +| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | +| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | +| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | +| `SAST_GITLEAKS_COMMIT_FROM` | | The commit a Gitleaks scan starts at. | +| `SAST_GITLEAKS_COMMIT_TO` | | The commit a Gitleaks scan ends at. | +| `SAST_GITLEAKS_HISTORIC_SCAN` | `false` | Flag to enable a historic Gitleaks scan. | #### Docker-in-Docker orchestrator @@ -315,11 +348,12 @@ The following variables configure the Docker-in-Docker orchestrator, and therefo Some analyzers can be customized with environment variables. -| Environment variable | Analyzer | Description | -|-----------------------------|----------|-------------| +| Environment variable | Analyzer | Description | +|---------------------------------------|----------------------|-------------| | `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | | `KUBESEC_HELM_CHARTS_PATH` | Kubesec | Optional path to Helm charts that `helm` will use to generate a Kubernetes manifest that `kubesec` will scan. If dependencies are defined, `helm dependency build` should be ran in a `before_script` to fetch the necessary dependencies. | | `KUBESEC_HELM_OPTIONS` | Kubesec | Additional arguments for the `helm` executable. | +| `COMPILE` | SpotBugs | Set to `false` to disable project compilation and dependency fetching. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195252) in GitLab 13.1. | | `ANT_HOME` | SpotBugs | The `ANT_HOME` environment variable. | | `ANT_PATH` | SpotBugs | Path to the `ant` executable. | | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | @@ -333,6 +367,7 @@ Some analyzers can be customized with environment variables. | `FAIL_NEVER` | SpotBugs | Set to `1` to ignore compilation failure. | | `SAST_GOSEC_CONFIG` | Gosec | Path to configuration for Gosec (optional). | | `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | +| `SEARCH_MAX_DEPTH` | any | Maximum number of directories traversed when searching for source code files. Default: `4`. | #### Custom environment variables @@ -494,7 +529,6 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/secrets:2 registry.gitlab.com/gitlab-org/security-products/analyzers/security-code-scan:2 registry.gitlab.com/gitlab-org/security-products/analyzers/sobelow:2 registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/tslint:2 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -509,7 +543,7 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set SAST CI job variables to use local SAST analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must replace -`SAST_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: +`SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: ```yaml include: diff --git a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png deleted file mode 100644 index 17893610f10..00000000000 Binary files a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png and /dev/null differ diff --git a/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png b/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png new file mode 100644 index 00000000000..4aa7dd83c8d Binary files /dev/null and b/doc/user/application_security/secret_detection/img/secret_detection_v13_2.png differ diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 85933c31a34..ea635212c5d 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -25,7 +25,7 @@ GitLab displays identified secrets as part of the SAST reports visibly in a few - Pipelines' **Security** tab - Report in the merge request widget -![Secret Detection in merge request widget](img/secret-detection-merge-request-ui.png) +![Secret Detection in merge request widget](img/secret_detection_v13_2.png) ## Use cases @@ -39,7 +39,8 @@ To run Secret Detection jobs, by default, you need GitLab Runner with the [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. If you're using the shared Runners on GitLab.com, this is enabled by default. -CAUTION: **Caution:** Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. +CAUTION: **Caution:** +Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** If you use your own Runners, make sure the Docker version installed @@ -118,15 +119,15 @@ declare a job with the same name as the SAST job to override. Place this new job inclusion and specify any additional keys under it. In the following example, we include the Secret Detection template and at the same time we -override the `secret-scan` job with the `SECRET_DETECTION_HISTORIC_SCAN` variable to `true`: +override the `secret_detection` job with the `SECRET_DETECTION_HISTORIC_SCAN` variable to `true`: ```yaml include: - template: Secret-Detection.gitlab-ci.yml -secrets-scan: +secret_detection: variables: - SECRET_DETECTION_HISTORIC_SCAN: true + SECRET_DETECTION_HISTORIC_SCAN: "true" ``` Because the template is [evaluated before](../../../ci/yaml/README.md#include) @@ -146,6 +147,16 @@ Secret Detection can be customized by defining available variables: | `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +### Logging Level + +You can control the verbosity of logs by setting the `SECURE_LOG_LEVEL` env var. The default is set to `info`, you can set it to any of the following levels: + +- `fatal` +- `error` +- `warn` +- `info` +- `debug` + ## Full History Secret Scan GitLab 12.11 introduced support for scanning the full history of a repository. This new functionality diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png index 0dfe7b637cd..d98fb71ae37 100644 Binary files a/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png and b/doc/user/application_security/security_dashboard/img/group_security_dashboard_export_csv_v13_1.png differ diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png deleted file mode 100644 index 4c7b5cc724f..00000000000 Binary files a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png and /dev/null differ diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png new file mode 100644 index 00000000000..d6cfc2de980 Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_2_noNav.png differ diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png deleted file mode 100644 index a500f186c2b..00000000000 Binary files a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png and /dev/null differ diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png new file mode 100644 index 00000000000..75b5ad1d885 Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png differ diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png deleted file mode 100644 index 670c90d10a3..00000000000 Binary files a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png and /dev/null differ diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png new file mode 100644 index 00000000000..591a08f4d7a Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png differ diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png new file mode 100644 index 00000000000..7cab7b0a61f Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png differ diff --git a/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png b/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png new file mode 100644 index 00000000000..9cf95b197fe Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/standalone_vulnerability_page_v13_1.png differ diff --git a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png new file mode 100644 index 00000000000..2b792727a99 Binary files /dev/null and b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png differ diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 60798b9c921..9a13d143d1f 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Secure +group: Threat Insights +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Security Dashboard **(ULTIMATE)** @@ -9,7 +12,7 @@ vulnerabilities in your groups, projects and pipelines. You can also drill down into a vulnerability and get extra information, see which project it comes from, the file it's in, and various metadata to help you analyze -the risk. You can also action these vulnerabilities by creating an issue for them, +the risk. You can also take actions on vulnerabilities by creating an issue for them, or by dismissing them. To benefit from the Security Dashboard you must first configure one of the @@ -42,7 +45,7 @@ At the pipeline level, the Security section displays the vulnerabilities present Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security findings. -![Pipeline Security Dashboard](img/pipeline_security_dashboard_v12_6.png) +![Pipeline Security Dashboard](img/pipeline_security_dashboard_v13_2.png) NOTE: **Note:** A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. @@ -51,56 +54,52 @@ A pipeline consists of multiple jobs, including SAST and DAST scanning. If any j > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6165) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.1. -At the project level, the Security Dashboard displays the latest security reports for your project. -Use it to find and fix vulnerabilities. +At the project level, the Security Dashboard displays the vulnerabilities merged into your project's +[default branch](../../project/repository/branches/index.md#default-branch). Access it by navigating +to **Security & Compliance > Security Dashboard**. -![Project Security Dashboard](img/project_security_dashboard_v13_0.png) +The Security Dashboard first displays the total number of vulnerabilities by severity (for example, +Critical, High, Medium, Low). Below this, a table displays each vulnerability's status, severity, +and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) +page to view more information about that vulnerability. -### Export vulnerabilities +You can filter the vulnerabilities by: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +- Status +- Severity +- Report type -You can export all your project's vulnerabilities as CSV by clicking on the export button located at top right of the Project Security Dashboard. This will initiate the process, and once complete, the CSV report will be downloaded. The report will contain all vulnerabilities in the project as filters won't apply. +You can also dismiss vulnerabilities in the table: -NOTE: **Note:** -It may take several minutes for the download to start if your project consists -of thousands of vulnerabilities. Do not close the page until the download finishes. +1. Select the checkbox for each vulnerability you want to dismiss. +1. In the menu that appears, select the reason for dismissal and click **Dismiss Selected**. -![CSV Export Button](img/project_security_dashboard_export_csv_v12_10.png) +![Project Security Dashboard](img/project_security_dashboard_v13_2.png) ## Group Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6709) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5. -The group Security Dashboard gives an overview of the vulnerabilities of all the -projects in a group and its subgroups. +The group Security Dashboard gives an overview of the vulnerabilities in the default branches of the +projects in a group and its subgroups. Access it by navigating to **Security > Security Dashboard** +for your group. -First, navigate to the Security Dashboard found under your group's -**Security** tab. +NOTE: **Note:** +The Security Dashboard only shows projects with [security reports](#supported-reports) enabled in a +group. + +![Dashboard with action buttons and metrics](img/group_security_dashboard_v13_2_noNav.png) -Once you're on the dashboard, at the top you should see a series of filters for: +You can filter which vulnerabilities the Security Dashboard displays by: - Status - Severity - Report type +- Project -NOTE: **Note:** -The dashboard only shows projects with [security reports](#supported-reports) enabled in a group. - -![Dashboard with action buttons and metrics](img/group_security_dashboard_v13_0.png) - -Selecting one or more filters will filter the results in this page. - -The main section is a list of all the vulnerabilities in the group, sorted by severity. -In that list, you can see the severity of the vulnerability, its name, its -confidence (likelihood of the vulnerability to be a positive one), and the project -it's from. - -If you hover over a row, the following actions appear: - -- More info -- Create issue -- Dismiss vulnerability +A table lists the vulnerabilities, sorted by severity. The table shows each vulnerability's status, +severity, and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) +page to view more information about that vulnerability. Next to the list is a timeline chart that shows how many open vulnerabilities your projects had at various points in time. You can filter among 30, 60, and @@ -120,28 +119,14 @@ vulnerabilities are not included either. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). -### Export vulnerabilities - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213013) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. - -You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** button -located at the top right of the **Group Security Dashboard**. After the report builds, the CSV -report downloads to your local machine. The report contains all vulnerabilities for the projects -defined in the **Group Security Dashboard**, as filters don't apply to the export function. - -NOTE: **Note:** -It may take several minutes for the download to start if your project contains thousands of -vulnerabilities. Don't close the page until the download finishes. - -![CSV Export Button](img/group_security_dashboard_export_csv_v13_1.png) - ## Instance Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6953) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.8. -At the instance level, the Security Dashboard displays the vulnerabilities -present in all of the projects that you have added to it. It includes all -of the features of the [group security dashboard](#group-security-dashboard). +At the instance level, the Security Dashboard displays the vulnerabilities present in the default +branches of all the projects you configure to display on the dashboard. It includes all the +[group Security Dashboard's](#group-security-dashboard) +features. You can access the Instance Security Dashboard from the menu bar at the top of the page. Under **More**, select **Security**. @@ -156,27 +141,25 @@ To add projects to the dashboard: 1. Search for and add one or more projects using the **Search your projects** field. 1. Click the **Add projects** button. -Once added, the dashboard will display the vulnerabilities found in your chosen -projects. +Once added, the Security Dashboard displays the vulnerabilities found in your chosen projects' +default branches. -![Instance Security Dashboard with projects](img/instance_security_dashboard_with_projects_v13_0.png) +![Instance Security Dashboard with projects](img/instance_security_dashboard_with_projects_v13_2_sm.png) -### Export vulnerabilities +## Export vulnerabilities -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. -You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** -button located at top right of the **Instance Security Dashboard**. After the report +You can export all your vulnerabilities in CSV format by clicking the **{upload}** **Export** +button located at top right of the **Security Dashboard**. After the report is built, the CSV report downloads to your local machine. The report contains all -vulnerabilities for the projects defined in the **Instance Security Dashboard**, +vulnerabilities for the projects defined in the **Security Dashboard**, as filters don't apply to the export function. NOTE: **Note:** It may take several minutes for the download to start if your project contains thousands of vulnerabilities. Do not close the page until the download finishes. -![CSV Export Button](img/instance_security_dashboard_export_csv_v13_0.png) - ## Keeping the dashboards up to date The Security Dashboard displays information from the results of the most recent @@ -194,12 +177,34 @@ Dashboard regardless of how often the default branch is updated. That way, reports are created even if no code change happens. +CAUTION: **Warning:** +Running Dependency Scanning from a scheduled pipeline might result in false negatives if your +project doesn't have a lock file and isn't configured for Continuous Delivery. A lock file is a file +that lists all transient dependencies and keeps track of their exact versions. The false negative +can occur because the dependency version resolved during the scan might differ from the ones +resolved when your project was built and released, in a previous pipeline. Java projects can't have +lock files. Python projects can have lock files, but GitLab Secure tools don't support them. + ## Security scans using Auto DevOps When using [Auto DevOps](../../../topics/autodevops/index.md), use [special environment variables](../../../topics/autodevops/customize.md#environment-variables) to configure daily security scans. +## Vulnerability list + +Each dashboard's vulnerability list contains vulnerabilities from the latest scans that were merged +into the default branch. +Click any vulnerability in the table to see more information on that vulnerability. To create an +issue associated with the vulnerability, click the **Create Issue** button. + +![Create an issue for the vulnerability](img/standalone_vulnerability_page_v13_1.png) + +Once you create the issue, the vulnerability list contains a link to the issue and an icon whose +color indicates the issue's status (green for open issues, blue for closed issues). + +![Display attached issues](img/vulnerability_list_table_v13_1.png) + - ## License list > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13582) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.7. @@ -696,13 +680,40 @@ and the associated classifications for each. Policies can be configured by maintainers of the project. -![Edit Policy](img/policies_maintainer_edit_v13_0.png) -![Add Policy](img/policies_maintainer_add_v13_0.png) +![Edit Policy](img/policies_maintainer_edit_v13_2.png) +![Add Policy](img/policies_maintainer_add_v13_2.png) Developers of the project can view the policies configured in a project. ![View Policies](img/policies_v13_0.png) +### Enabling License Approvals within a project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. + +`License-Check` is an approval rule you can enable to allow an approver, individual, or group to +approve a merge request that contains a `denied` license. + +You can enable `License-Check` one of two ways: + +- Create a [project approval rule](../../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) + with the case-sensitive name `License-Check`. +- Create an approval group in the [project policies section for License Compliance](#policies). + You must set this approval group's number of approvals required to greater than zero. Once you + enable this group in your project, the approval rule is enabled for all merge requests. + +Any code changes cause the approvals required to reset. + +An approval is required when a license report: + +- Contains a dependency that includes a software license that is `denied`. +- Is not generated during pipeline execution. + +An approval is optional when a license report: + +- Contains no software license violations. +- Contains only new licenses that are `allowed` or unknown. + ## Troubleshooting ### `ERROR -- : asdf: No preset version installed for command` diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 44802214d7b..599f46b2c40 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -487,14 +487,15 @@ For example, to customize the commit message to output NOTE: **Note:** Custom commit messages for each applied Suggestion (and for batch Suggestions) will be -introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/issues/25381). +introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/-/issues/25381). ### Batch Suggestions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/25486) in GitLab 13.1 as an [alpha feature](https://about.gitlab.com/handbook/product/#alpha). -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-batch-suggestions). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [alpha feature](https://about.gitlab.com/handbook/product/#alpha). +> - It was deployed behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) on GitLab 13.2. +> - It's enabled on GitLab.com. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-batch-suggestions-core-only). You can apply multiple suggestions at once to reduce the number of commits added to your branch to address your reviewers' requests. @@ -515,25 +516,25 @@ to your branch to address your reviewers' requests. ![A code change suggestion displayed, with the button to apply the batch of suggestions highlighted.](img/apply_batch_of_suggestions_v13_1.jpg "Apply a batch of suggestions") -#### Enable or disable Batch Suggestions +#### Enable or disable Batch Suggestions **(CORE ONLY)** Batch Suggestions is -deployed behind a feature flag that is **disabled by default**. +deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it for your instance. +can opt to disable it for your instance. To enable it: ```ruby # Instance-wide -Feature.enable(:batched_suggestions) +Feature.enable(:batch_suggestions) ``` To disable it: ```ruby # Instance-wide -Feature.disable(:batched_suggestions) +Feature.disable(:batch_suggestions) ``` ## Start a thread by replying to a standard comment diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 6522f5c4053..bcaba97cab7 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -78,31 +78,34 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | Setting | GitLab.com | Default | | ----------- | ----------------- | ------------- | | Artifacts maximum size (uncompressed) | 1G | 100M | -| Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | kept forever | deleted after 30 days unless otherwise specified | +| Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | | Scheduled Pipeline Cron | `*/5 * * * *` | `19 * * * *` | | [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited | [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited | | [Max number of instance level variables](../../administration/instance_limits.md#number-of-instance-level-variables) | `25` | `25` | +| [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs-core-only) | 3 months | Never | ## Repository size limit -The maximum size your Git repository is allowed to be, including LFS. If you are near -or over the size limit, you can [reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). +GitLab.com has the following [account limits](../admin_area/settings/account_and_limit_settings.md) enabled. If a setting is not listed, it is set to the default value. -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ------------- | -| Repository size including LFS | 10G | Unlimited | +If you are near +or over the repository size limit, you can [reduce your repository size with Git](../project/repository/reducing_the_repo_size_using_git.md). + +| Setting | GitLab.com | Default | +| ----------- | ----------- | ------------- | +| Repository size including LFS | 10 GB | Unlimited | NOTE: **Note:** -`git push` and GitLab project imports are limited to 5GB per request. Git LFS and imports other than a file upload are not affected by this limit. +`git push` and GitLab project imports are limited to 5 GB per request through Cloudflare. Git LFS and imports other than a file upload are not affected by this limit. ## IP range GitLab.com is using the IP range `34.74.90.64/28` for traffic from its Web/API fleet. This whole range is solely allocated to GitLab. You can expect connections from webhooks or repository mirroring to come -from those IPs and whitelist them. +from those IPs and allow them. -GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com you might need to whitelist CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)) +GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)). For outgoing connections from CI/CD runners we are not providing static IP addresses. All our runners are deployed into Google Cloud Platform (GCP) - any IP based @@ -334,9 +337,9 @@ Windows Shared Runners: ```yaml .shared_windows_runners: tags: - - shared-windows - - windows - - windows-1809 + - shared-windows + - windows + - windows-1809 stages: - build @@ -349,17 +352,17 @@ before_script: build: extends: - - .shared_windows_runners + - .shared_windows_runners stage: build script: - - echo "running scripts in the build job" + - echo "running scripts in the build job" test: extends: - - .shared_windows_runners + - .shared_windows_runners stage: test script: - - echo "running scripts in the test job" + - echo "running scripts in the test job" ``` #### Limitations and known issues @@ -581,9 +584,9 @@ is used to forward logs to an [Elastic cluster](https://gitlab.com/gitlab-com/ru You can view more information in our runbooks such as: -- A [detailed list of what we're logging](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#what-are-we-logging) -- Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#retention) -- A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#logging-infrastructure-overview) +- A [detailed list of what we're logging](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#what-are-we-logging) +- Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#retention) +- A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/-/tree/master/docs/logging#logging-infrastructure-overview) ## GitLab.com at scale diff --git a/doc/user/group/bulk_editing/img/bulk-editing.png b/doc/user/group/bulk_editing/img/bulk-editing.png deleted file mode 100644 index ca1662a781b..00000000000 Binary files a/doc/user/group/bulk_editing/img/bulk-editing.png and /dev/null differ diff --git a/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png b/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png new file mode 100644 index 00000000000..9f28fabdf0c Binary files /dev/null and b/doc/user/group/bulk_editing/img/bulk-editing_v13_2.png differ diff --git a/doc/user/group/bulk_editing/index.md b/doc/user/group/bulk_editing/index.md index c4ccc1f7b52..35bdc6696eb 100644 --- a/doc/user/group/bulk_editing/index.md +++ b/doc/user/group/bulk_editing/index.md @@ -8,12 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: **Note:** Bulk editing issues and merge requests is also available at the **project level**. -For more details, see [Bulk editing issues, epics, and merge requests](../../project/bulk_editing.md). +For more details, see [Bulk editing issues and merge requests at the project level](../../project/bulk_editing.md). If you want to update attributes across multiple issues, epics, or merge requests in a group, you can do it by bulk editing them, that is, editing them together. -![Bulk editing](img/bulk-editing.png) +![Bulk editing](img/bulk-editing_v13_2.png) ## Bulk edit issues at the group level @@ -24,8 +24,12 @@ You need a permission level of [Reporter or higher](../../permissions.md) to man When bulk editing issues in a group, you can edit the following attributes: +- Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in + [GitLab Premium](https://about.gitlab.com/pricing/) 13.2.) **(PREMIUM)** - Milestone - Labels +- Health status ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218395) in + [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2.) **(ULTIMATE)** To update multiple project issues at the same time: diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 5cdac7ae892..89e0c4898fb 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -38,10 +38,11 @@ the project. In the case of sub-groups, GitLab uses the cluster of the closest ancestor group to the project, provided the cluster is not disabled. -## Multiple Kubernetes clusters **(PREMIUM)** +## Multiple Kubernetes clusters -With [GitLab Premium](https://about.gitlab.com/pricing/premium/), you can associate -more than one Kubernetes cluster to your group, and maintain different clusters +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab Core in 13.2. + +You can associate more than one Kubernetes cluster to your group, and maintain different clusters for different environments, such as development, staging, and production. When adding another cluster, @@ -93,7 +94,7 @@ To clear the cache: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8. Domains at the cluster level permit support for multiple domains -per [multiple Kubernetes clusters](#multiple-kubernetes-clusters-premium). When specifying a domain, +per [multiple Kubernetes clusters](#multiple-kubernetes-clusters) When specifying a domain, this will be automatically set as an environment variable (`KUBE_INGRESS_BASE_DOMAIN`) during the [Auto DevOps](../../../topics/autodevops/index.md) stages. @@ -127,8 +128,8 @@ And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/RE ```yaml stages: -- test -- deploy + - test + - deploy test: stage: test diff --git a/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png b/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png new file mode 100644 index 00000000000..c7e1448fea9 Binary files /dev/null and b/doc/user/group/epics/img/epic_activity_sort_order_v13_2.png differ diff --git a/doc/user/group/epics/img/epics_list_view_v12.5.png b/doc/user/group/epics/img/epics_list_view_v12.5.png deleted file mode 100644 index 6e3c39009be..00000000000 Binary files a/doc/user/group/epics/img/epics_list_view_v12.5.png and /dev/null differ diff --git a/doc/user/group/epics/img/new_epic_form_v13.2.png b/doc/user/group/epics/img/new_epic_form_v13.2.png new file mode 100644 index 00000000000..3d24763d105 Binary files /dev/null and b/doc/user/group/epics/img/new_epic_form_v13.2.png differ diff --git a/doc/user/group/epics/img/new_epic_from_groups_v13.2.png b/doc/user/group/epics/img/new_epic_from_groups_v13.2.png new file mode 100644 index 00000000000..85bc4255595 Binary files /dev/null and b/doc/user/group/epics/img/new_epic_from_groups_v13.2.png differ diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 85164292265..a2b04e2d7fe 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -14,8 +14,15 @@ Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. - -![epics list view](img/epics_list_view_v12.5.png) +An epic's page contains the following tabs: + +- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are + shown in a tree view. + - Click the chevron (**>**) next to a parent epic to reveal the child epics and issues. + - Hover over the total counts to see a breakdown of open and closed items. +- **Roadmap**: a roadmap view of child epics which have start and due dates. + +![epic view](img/epic_view_v13.0.png) ## Use cases @@ -28,6 +35,7 @@ milestones. To learn what you can do with an epic, see [Manage epics](manage_epics.md). Possible actions include: - [Create an epic](manage_epics.md#create-an-epic) +- [Edit an epic](manage_epics.md#edit-an-epic) - [Bulk-edit epics](../bulk_editing/index.md#bulk-edit-epics) - [Delete an epic](manage_epics.md#delete-an-epic) - [Close an epic](manage_epics.md#close-an-epic) @@ -165,6 +173,19 @@ Once you write your comment, you can either: - Click **Comment**, and your comment will be published. - Click **Start thread**, and you will start a thread within that epic's discussion. +### Activity sort order + +> [Introduced](https://https://gitlab.com/gitlab-org/gitlab/-/issues/214364) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +You can reverse the default order and interact with the activity feed sorted by most recent items +at the top. Your preference is saved via local storage and automatically applied to every issue +you view. + +To change the activity sort order, click the **Oldest first** dropdown menu and select either oldest +or newest items to be shown first. + +![Issue activity sort order dropdown button](img/epic_activity_sort_order_v13_2.png) + ## Award emoji You can [award an emoji](../../award_emojis.md) to that epic or its comments. diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 26d5cb51e01..4f9bb0e24fb 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -18,12 +18,42 @@ A paginated list of epics is available in each group from where you can create a new epic. The list of epics includes also epics from all subgroups of the selected group. From your group page: -1. Go to **Epics**. +### Create an epic from the epic list + +To create an epic from the epic list, in a group: + +1. Go to **{epic}** **Epics**. 1. Click **New epic**. 1. Enter a descriptive title. 1. Click **Create epic**. -You will be taken to the new epic where can edit the following details: +### Access the New Epic form + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211533) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +There are two ways to get to the New Epic form and create an epic in the group you're in: + +- From an epic in your group, click **New Epic**. +- From anywhere, in the top menu, click **plus** (**{plus-square}**) **> New epic**. + + ![New epic from an open epic](img/new_epic_from_groups_v13.2.png) + +### Elements of the New Epic form + +When you're creating a new epic, these are the fields you can fill in: + +- Title +- Description +- Confidentiality checkbox +- Labels +- Start date +- Due date + +![New epic form](img/new_epic_form_v13.2.png) + +## Edit an epic + +After you create an epic, you can edit change the following details: - Title - Description @@ -31,15 +61,16 @@ You will be taken to the new epic where can edit the following details: - Due date - Labels -An epic's page contains the following tabs: +To edit an epic's title or description: -- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are - shown in a tree view. - - Click the > beside a parent epic to reveal the child epics and issues. - - Hover over the total counts to see a breakdown of open and closed items. -- **Roadmap**: a roadmap view of child epics which have start and due dates. +1. Click the **Edit title and description** **{pencil}** button. +1. Make your changes. +1. Click **Save changes**. -![epic view](img/epic_view_v13.0.png) +To edit an epics' start date, due date, or labels: + +1. Click **Edit** next to each section in the epic sidebar. +1. Select the dates or labels for your epic. ## Bulk-edit epics @@ -118,27 +149,17 @@ The sort option and order is saved and used wherever you browse epics, including ## Make an epic confidential -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213068) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-confidential-epics-premium-only). **(PREMIUM ONLY)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213068) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0 behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/224513) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. When you're creating an epic, you can make it confidential by selecting the **Make this epic confidential** checkbox. -### Enable Confidential Epics **(PREMIUM ONLY)** +### Disable confidential epics **(PREMIUM ONLY)** -The Confidential Epics feature is under development and not ready for production use. -It's deployed behind a feature flag that is **disabled by default**. +The confidential epics feature is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. - -To enable it: - -```ruby -Feature.enable(:confidential_epics) -``` +can disable it for your self-managed instance. To disable it: @@ -233,7 +254,7 @@ To move an issue to another epic: > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) to [GitLab Premium](https://about.gitlab.com/pricing/) in 12.8. If you have the necessary [permissions](../../permissions.md) to close an issue and create an -epic in the parent group, you can promote an issue to an epic with the `/promote` +epic in the immediate parent group, you can promote an issue to an epic with the `/promote` [quick action](../../project/quick_actions.md#quick-actions-for-issues-merge-requests-and-epics). Only issues from projects that are in groups can be promoted. When attempting to promote a confidential issue, a warning will display. Promoting a confidential issue to an epic will make all information diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 324c912b2be..5ba0680127c 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -31,7 +31,7 @@ Each group on the **Groups** page is listed with: - How many subgroups it has. - How many projects it contains. -- How many members the group has, not including members inherited from parent groups. +- How many members the group has, not including members inherited from parent group(s). - The group's visibility. - A link to the group's settings, if you have sufficient permissions. - A link to leave the group, if you are a member. @@ -184,6 +184,33 @@ of a group: 1. Give a different member **Owner** permissions. 1. Have the new owner sign in and remove **Owner** permissions from you. +## Remove a member from the group + +Only users with permissions of [Owner](../permissions.md#group-members-permissions) can manage +group members. + +You can remove a member from the group if the given member has a direct membership in the group. If +membership is inherited from a parent group, then the member can be removed only from the parent +group itself. + +When removing a member, you can decide whether to unassign the user from all issues and merge +requests they are currently assigned or leave the assignments as they are. + +- **Unassigning the removed member** from all issues and merge requests might be helpful when a user + is leaving a private group and you wish to revoke their access to any issues and merge requests + they are assigned. +- **Keeping the issues and merge requests assigned** might be helpful for groups that accept public + contributions where a user doesn't have to be a member to be able to contribute to issues and + merge requests. + +To remove a member from a group: + +1. In a group, go to **{users}** **Members**. +1. Click the **Delete** **{remove}** button next to a group member you want to remove. + A **Remove member** modal appears. +1. (Optional) Select the **Also unassign this user from related issues and merge requests** checkbox. +1. Click **Remove member**. + ## Changing the default branch protection of a group > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7583) in GitLab 12.9. @@ -397,7 +424,7 @@ When transferring groups, note: - Changing a group's parent can have unintended side effects. See [Redirects when changing repository paths](../project/index.md#redirects-when-changing-repository-paths). - You can only transfer groups to groups you manage. - You must update your local repositories to point to the new location. -- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. +- If the immediate parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. - Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. ## Group settings @@ -435,7 +462,7 @@ It is currently not possible to rename a namespace if it contains a project with [Container Registry](../packages/container_registry/index.md) tags, because the project cannot be moved. -TIP: **TIP:** +TIP: **Tip:** If you want to retain ownership over the original namespace and protect the URL redirects, then instead of changing a group's path or renaming a username, you can create a new group and transfer projects to it. @@ -516,7 +543,7 @@ underlying projects, issues, etc, by IP address. This can help ensure that particular content doesn't leave the premises, while not blocking off access to the entire instance. -Add one or more allowed IP subnets using CIDR notation in comma separated format to the group settings and anyone +Add one or more allowed IP subnets using CIDR notation to the group settings and anyone coming from a different IP address won't be able to access the restricted content. @@ -529,6 +556,12 @@ Restriction currently applies to: To avoid accidental lock-out, admins and group owners are able to access the group regardless of the IP restriction. +To enable this feature: + +1. Navigate to the group’s **Settings > General** page. +1. Expand the **Permissions, LFS, 2FA** section, and enter IP address ranges into **Allow access to the following IP addresses** field. +1. Click **Save changes**. + #### Allowed domain restriction **(PREMIUM)** >- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7297) in [GitLab Premium and Silver](https://about.gitlab.com/pricing/) 12.2. @@ -554,7 +587,7 @@ Some domains cannot be restricted. These are the most popular public email domai To enable this feature: 1. Navigate to the group's **Settings > General** page. -1. Expand the **Permissions, LFS, 2FA** section, and enter the domain names into **Restrict membership by email** field. You can enter multiple domains by separating each domain with a comma (,). +1. Expand the **Permissions, LFS, 2FA** section, and enter the domain names into **Restrict membership by email** field. 1. Click **Save changes**. This will enable the domain-checking for all new users added to the group from this moment on. @@ -571,9 +604,9 @@ You can only choose projects in the group as the template source. This includes projects shared with the group, but it **excludes** projects in subgroups or parent groups of the group being configured. -You can configure this feature for both subgroups and parent groups. A project +You can configure this feature for both subgroups and immediate parent groups. A project in a subgroup will have access to the templates for that subgroup, as well as -any parent groups. +any immediate parent groups. ![Group file template dropdown](img/group_file_template_dropdown.png) @@ -617,6 +650,23 @@ To enable this feature: 1. Expand the **Permissions, LFS, 2FA** section, and select **Disable group mentions**. 1. Click **Save changes**. +#### Enabling delayed Project removal **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. + +By default, projects within a group are deleted immediately. +Optionally, on [Premium or Silver](https://about.gitlab.com/pricing/) or higher tiers, +you can configure the projects within a group to be deleted after a delayed interval. + +During this interval period, the projects will be in a read-only state and can be restored, if required. +The interval period defaults to 7 days, and can be modified by an admin in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). + +To enable delayed deletion of projects: + +1. Navigate to the group's **Settings > General** page. +1. Expand the **Permissions, LFS, 2FA** section, and check **Enable delayed project removal**. +1. Click **Save changes**. + ### Advanced settings - **Projects**: View all projects within that group, add members to each project, diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 2704147dcdd..bc9d228011a 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -8,11 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Iterations **(STARTER)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214713) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.1. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's able to be enabled or disabled per-group -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-iterations-core-only). **(CORE ONLY)** +> - It was deployed behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/221047) on GitLab 13.2. +> - It's enabled on GitLab.com. +> - It's able to be enabled or disabled per-group. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-iterations-core-only). **(CORE ONLY)** Iterations are a way to track issues over a period of time. This allows teams to track velocity and volatility metrics. Iterations can be used with [milestones](../../project/milestones/index.md) @@ -38,7 +39,7 @@ From there you can create a new iteration or click an iteration to get a more de ## Create an iteration NOTE: **Note:** -A permission level of [Developer or higher](../../permissions.md) is required to create iterations. +You need Developer [permissions](../../permissions.md) or higher to create an iteration. To create an iteration: @@ -47,12 +48,20 @@ To create an iteration: 1. Enter the title, a description (optional), a start date, and a due date. 1. Click **Create iteration**. The iteration details page opens. -### Enable Iterations **(CORE ONLY)** +## Edit an iteration -GitLab Iterations feature is under development and not ready for production use. -It is deployed behind a feature flag that is **disabled by default**. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218277) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.2. + +NOTE: **Note:** +You need Developer [permissions](../../permissions.md) or higher to edit an iteration. + +To edit an iteration, click the three-dot menu (**{ellipsis_v}**) > **Edit iteration**. + +## Disable Iterations **(CORE ONLY)** + +GitLab Iterations feature is deployed with a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. `:group_iterations` can be enabled or disabled per-group. +can disable it for your instance. `:group_iterations` can be enabled or disabled per-group. To enable it: diff --git a/doc/user/group/roadmap/img/roadmap_view_v13_0.png b/doc/user/group/roadmap/img/roadmap_view_v13_0.png deleted file mode 100644 index a5b76b84418..00000000000 Binary files a/doc/user/group/roadmap/img/roadmap_view_v13_0.png and /dev/null differ diff --git a/doc/user/group/roadmap/img/roadmap_view_v13_2.png b/doc/user/group/roadmap/img/roadmap_view_v13_2.png new file mode 100644 index 00000000000..b4f889afaa4 Binary files /dev/null and b/doc/user/group/roadmap/img/roadmap_view_v13_2.png differ diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index 614ed700cfc..c185055f6b2 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -12,22 +12,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/5164) and later, the epic bars show epics' title, progress, and completed weight percentage. > - Milestones appear in roadmaps in [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/6802), and later. > - Feature flag for milestones visible in roadmaps removed in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29641). +> - In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/214375) and later, the Roadmap also shows milestones in projects in a group. +> - In [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/212494) and later, milestone bars can be collapsed and expanded. -Epics and milestones within a group containing **Start date** and/or **Due date** -can be visualized in a form of a timeline (that is, a Gantt chart). The Roadmap page -shows such a visualization for all the epics and milestones which are under a group or one of its -subgroups. +Epics and milestones within a group containing a start date or due date can be visualized in a form +of a timeline (that is, a Gantt chart). The Roadmap page shows the epics and milestones in a +group, one of its subgroups, or a project in one of the groups. On the epic bars, you can see the each epic's title, progress, and completed weight percentage. When you hover over an epic bar, a popover appears with the epic's title, start date, due date, and weight completed. You can expand epics that contain child epics to show their child epics in the roadmap. -You can click the chevron **{chevron-down}** next to the epic title to expand and collapse the child epics. +You can click the chevron (**{chevron-down}**) next to the epic title to expand and collapse the child epics. On top of the milestone bars, you can see their title. When you hover a milestone bar or title, a popover appears with its title, start date and due date. +You can also click the chevron (**{chevron-down}**) next to the **Milestones** heading to toggle the list of the milestone bars. -![roadmap view](img/roadmap_view_v13_0.png) +![roadmap view](img/roadmap_view_v13_2.png) A dropdown menu allows you to show only open or closed epics. By default, all epics are shown. diff --git a/doc/user/group/saml_sso/group_managed_accounts.md b/doc/user/group/saml_sso/group_managed_accounts.md new file mode 100644 index 00000000000..e317573d89d --- /dev/null +++ b/doc/user/group/saml_sso/group_managed_accounts.md @@ -0,0 +1,116 @@ +--- +type: reference, howto +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Group Managed Accounts **(PREMIUM)** + +CAUTION: **Warning:** +This is a [Closed Beta](https://about.gitlab.com/handbook/product/#closed-beta) feature. + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/709) in GitLab 12.1. +> - It's deployed behind a feature flag, disabled by default. + +When [SSO for Groups](index.md) is being enforced, groups can enable an additional level of protection by enforcing the creation of dedicated user accounts to access the group. + +With group-managed accounts enabled, users are required to create a new, dedicated user linked to the group. +The notification email address associated with the user is locked to the email address received from the configured identity provider. +Without group-managed accounts, users can link their SAML identity with any existing user on the instance. + +When this option is enabled: + +- All users in the group will be required to log in via the SSO URL associated with the group. +- After the group-managed account has been created, group activity will require the use of this user account. +- Users can't share a project in the group outside the top-level group (also applies to forked projects). + +Upon successful authentication, GitLab prompts the user with options, based on the email address received from the configured identity provider: + +- To create a unique account with the newly received email address. +- If the received email address matches one of the user's verified GitLab email addresses, the option to convert the existing account to a group-managed account. ([Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/13481).) + +Since use of the group-managed account requires the use of SSO, users of group-managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: + +- The user will be unable to access the group (their credentials will no longer work on the identity provider when prompted to SSO). +- Contributions in the group (e.g. issues, merge requests) will remain intact. + +## Assertions + +When using group-managed accounts, the following user details need to be passed to GitLab as SAML +assertions to be able to create a user. + +| Field | Supported keys | +|-----------------|----------------| +| Email (required)| `email`, `mail` | +| Full Name | `name` | +| First Name | `first_name`, `firstname`, `firstName` | +| Last Name | `last_name`, `lastname`, `lastName` | + +## Feature flag **(PREMIUM ONLY)** + +Currently the group-managed accounts feature is behind a feature flag: `group_managed_accounts`. The flag is disabled by default. +To activate the feature, ask a GitLab administrator with Rails console access to run: + +```ruby +Feature.enable(:group_managed_accounts) +``` + +## Project restrictions for Group-managed accounts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12420) in GitLab 12.9. + +Projects within groups with enabled group-managed accounts are not to be shared with: + +- Groups outside of the parent group. +- Members who are not users managed by this group. + +This restriction also applies to projects forked from or to those groups. + +## Outer forks restriction for Group-managed accounts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34648) in GitLab 12.9. + +Groups with group-managed accounts can disallow forking of projects to destinations outside the group. +To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. +When enabled, projects within the group can only be forked to other destinations within the group (including its subgroups). + +## Credentials inventory for Group-managed accounts **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38133) in GitLab 12.8. + +Owners who manage user accounts in a group can view the following details of personal access tokens and SSH keys: + +- Owners +- Scopes +- Usage patterns + +To access the Credentials inventory of a group, navigate to **{shield}** **Security & Compliance > Credentials** in your group's sidebar. + +This feature is similar to the [Credentials inventory for self-managed instances](../../admin_area/credentials_inventory.md). + +## Limiting lifetime of personal access tokens of users in Group-managed accounts **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118893) in GitLab 12.10. + +Users in a group managed account can optionally specify an expiration date for +[personal access tokens](../../profile/personal_access_tokens.md). +This expiration date is not a requirement, and can be set to any arbitrary date. + +Since personal access tokens are the only token needed for programmatic access to GitLab, organizations with security requirements may want to enforce more protection to require regular rotation of these tokens. + +### Setting a limit + +Only a GitLab administrator or an owner of a Group-managed account can set a limit. Leaving it empty means that the [instance level restrictions](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only) on the lifetime of personal access tokens will apply. + +To set a limit on how long personal access tokens are valid for users in a group managed account: + +1. Navigate to the **{settings}** **Settings > General** page in your group's sidebar. +1. Expand the **Permissions, LFS, 2FA** section. +1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. +1. Click **Save changes**. + +Once a lifetime for personal access tokens is set, GitLab will: + +- Apply the lifetime for new personal access tokens, and require users managed by the group to set an expiration date that is no later than the allowed lifetime. +- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the allowed lifetime. Three hours is given to allow administrators/group owner to change the allowed lifetime, or remove it, before revocation takes place. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 81684038dc2..afd676cf897 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -5,32 +5,25 @@ group: Access info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# SAML SSO for GitLab.com groups **(SILVER ONLY)** +# SAML SSO for GitLab.com groups **(PREMIUM)** -> Introduced in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.0. +> Introduced in GitLab 11.0. -SAML on GitLab.com allows users to be added to a group. Those users can then sign in to GitLab.com. If such users don't already have an account on the GitLab instance, they can create one when signing in for the first time. +This page describes SAML for Groups. For instance-wide SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -If you follow our guidance to automate user provisioning using [SCIM](scim_setup.md) or [group-managed accounts](#group-managed-accounts), you do not need to create such accounts manually. +SAML on GitLab.com allows users to sign in through their SAML identity provider. If the user is not already a member, the sign-in process automatically adds the user to the appropriate group. -User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). +If you follow our guidance to automate user provisioning using [SCIM](scim_setup.md) or [group-managed accounts](group_managed_accounts.md), you do not need to create such accounts manually. -## Important notes - -Note the following: - -- This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab - instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -- SAML SSO for GitLab.com groups requires SCIM to sync users between providers. If a - group is not using SCIM, group Owners will still need to manage user accounts (for example, - removing users when necessary). +User synchronization of SAML SSO groups is supported through [SCIM](scim_setup.md). SCIM supports adding and removing users from the GitLab group. +For example, if you remove a user from the SCIM app, SCIM removes that same user from the GitLab group. ## Configuring your Identity Provider 1. Navigate to the group and click **Settings > SAML SSO**. -1. Configure your SAML server using the **Assertion consumer service URL** and **Identifier**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [your identity provider's documentation](#providers) for more details. +1. Configure your SAML server using the **Assertion consumer service URL**, **Identifier**, and **GitLab single sign-on URL**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [specific identity provider documentation](#providers) for more details. 1. Configure the SAML response to include a NameID that uniquely identifies each user. -1. Configure required assertions using the [table below](#assertions). +1. Configure [required assertions](group_managed_accounts.md#assertions) if using [Group Managed Accounts](group_managed_accounts.md). 1. Once the identity provider is set up, move on to [configuring GitLab](#configuring-gitlab). ![Issuer and callback for configuring SAML identity provider with GitLab.com](img/group_saml_configuration_information.png) @@ -55,123 +48,6 @@ Once users have signed into GitLab using the SSO SAML setup, changing the `NameI We recommend setting the NameID format to `Persistent` unless using a field (such as email) that requires a different format. -### SSO enforcement - -- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. -- [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. - -With this option enabled, users must use your group's GitLab single sign on URL to be added to the group or be added via SCIM. Users cannot be added manually, and may only access project/group resources via the UI by signing in through the SSO URL. - -However, users will not be prompted to log via SSO on each visit. GitLab will check whether a user has authenticated through the SSO link, and will only prompt the user to login via SSO if the session has expired. - -We intend to add a similar SSO requirement for [Git and API activity](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in the future. - -When SSO enforcement is enabled for a group, users cannot share a project in the group outside the top-level group, even if the project is forked. - -#### Group-managed accounts - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/709) in GitLab 12.1. - -When SSO is being enforced, groups can enable an additional level of protection by enforcing the creation of dedicated user accounts to access the group. - -Without group-managed accounts, users can link their SAML identity with any existing user on the instance. With group-managed accounts enabled, users are required to create a new, dedicated user linked to the group. The notification email address associated with the user is locked to the email address received from the configured identity provider. - -When this option is enabled: - -- All existing and new users in the group will be required to log in via the SSO URL associated with the group. -- After the group-managed account has been created, group activity will require the use of this user account. -- Users can't share a project in the group outside the top-level group (also applies to forked projects). - -Upon successful authentication, GitLab prompts the user with options, based on the email address received from the configured identity provider: - -- To create a unique account with the newly received email address. -- If the received email address matches one of the user's verified GitLab email addresses, the option to convert the existing account to a group-managed account. ([Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/13481).) - -Since use of the group-managed account requires the use of SSO, users of group-managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: - -- The user will be unable to access the group (their credentials will no longer work on the identity provider when prompted to SSO). -- Contributions in the group (e.g. issues, merge requests) will remain intact. - -##### Feature flag - -Currently the group-managed accounts feature is behind a feature flag: `group_managed_accounts`. The flag is disabled by default. -To activate the feature, ask a GitLab administrator with Rails console access to run: - -```ruby -Feature.enable(:group_managed_accounts) -``` - -##### Credentials inventory for Group-managed accounts **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38133) in GitLab 12.8. - -Owners who manage user accounts in a group can view the following details of personal access tokens and SSH keys: - -- Owners -- Scopes -- Usage patterns - -To access the Credentials inventory of a group, navigate to **{shield}** **Security & Compliance > Credentials** in your group's sidebar. - -This feature is similar to the [Credentials inventory for self-managed instances](../../admin_area/credentials_inventory.md). - -##### Limiting lifetime of personal access tokens of users in Group-managed accounts **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118893) in GitLab 12.10. - -Users in a group managed account can optionally specify an expiration date for -[personal access tokens](../../profile/personal_access_tokens.md). -This expiration date is not a requirement, and can be set to any arbitrary date. - -Since personal access tokens are the only token needed for programmatic access to GitLab, organizations with security requirements may want to enforce more protection to require regular rotation of these tokens. - -###### Setting a limit - -Only a GitLab administrator or an owner of a Group-managed account can set a limit. Leaving it empty means that the [instance level restrictions](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only) on the lifetime of personal access tokens will apply. - -To set a limit on how long personal access tokens are valid for users in a group managed account: - -1. Navigate to the **{settings}** **Settings > General** page in your group's sidebar. -1. Expand the **Permissions, LFS, 2FA** section. -1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. -1. Click **Save changes**. - -Once a lifetime for personal access tokens is set, GitLab will: - -- Apply the lifetime for new personal access tokens, and require users managed by the group to set an expiration date that is no later than the allowed lifetime. -- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the allowed lifetime. Three hours is given to allow administrators/group owner to change the allowed lifetime, or remove it, before revocation takes place. - -##### Outer forks restriction for Group-managed accounts - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34648) in GitLab 12.9. - -Groups with group-managed accounts can disallow forking of projects to destinations outside the group. -To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. -When enabled, projects within the group can only be forked to other destinations within the group (including its subgroups). - -##### Other restrictions for Group-managed accounts - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12420) in GitLab 12.9. - -Projects within groups with enabled group-managed accounts are not to be shared with: - -- Groups outside of the parent group. -- Members who are not users managed by this group. - -This restriction also applies to projects forked from or to those groups. - -#### Assertions - -When using group-managed accounts, the following user details need to be passed to GitLab as SAML -assertions to be able to create a user. - -| Field | Supported keys | -|-----------------|----------------| -| Email (required)| `email`, `mail` | -| Full Name | `name` | -| First Name | `first_name`, `firstname`, `firstName` | -| Last Name | `last_name`, `lastname`, `lastName` | - ### Metadata configuration GitLab provides metadata XML that can be used to configure your Identity Provider. @@ -185,7 +61,7 @@ GitLab provides metadata XML that can be used to configure your Identity Provide Once you've set up your identity provider to work with GitLab, you'll need to configure GitLab to use it for authentication: 1. Navigate to the group's **Settings > SAML SSO**. -1. Find the SSO URL from your Identity Provider and enter it the **Identity provider single sign on URL** field. +1. Find the SSO URL from your Identity Provider and enter it the **Identity provider single sign-on URL** field. 1. Find and enter the fingerprint for the SAML token signing certificate in the **Certificate** field. 1. Click the **Enable SAML authentication for this group** toggle switch. 1. Click the **Save changes** button. @@ -193,42 +69,27 @@ Once you've set up your identity provider to work with GitLab, you'll need to co ![Group SAML Settings for GitLab.com](img/group_saml_settings.png) NOTE: **Note:** -Please note that the certificate [fingerprint algorithm](#additional-setup-options) must be in SHA1. When configuring the identity provider, use a secure [signature algorithm](#additional-setup-options). - -## User access and management - -Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, please see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup). - -When a user tries to sign in with Group SSO, they'll need an account that's configured with one of the following: +Please note that the certificate [fingerprint algorithm](#additional-providers-and-setup-options) must be in SHA1. When configuring the identity provider, use a secure signature algorithm. -- [SCIM](scim_setup.md). -- [Group-managed accounts](#group-managed-accounts). -- A GitLab.com account. - -1. Click on the GitLab app in the identity provider's dashboard, or visit the Group's GitLab SSO URL. -1. Sign in to GitLab.com. The next time you connect on the same browser, you won't have to sign in again provided the active session has not expired. -1. Click on the **Authorize** button. - -On subsequent visits, users can access the group through the identify provider's dashboard or by visiting links directly. With the **enforce SSO** option turned on, users will be redirected to log in through the identity provider as required. - -### Role +### SSO enforcement -Upon first sign in, a new user is added to the parent group with the Guest role. Existing members with an appropriate role will have to elevate users to a higher role where relevant. +- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5291) in GitLab 11.8. +- [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9255) in GitLab 11.11 with ongoing enforcement in the GitLab UI. -If a user is already a member of the group, linking the SAML identity does not change their role. +With this option enabled, users must go through your group's GitLab single sign-on URL. They may also be added via SCIM, if configured. Users cannot be added manually, and may only access project/group resources via the UI by signing in through the SSO URL. -### Blocking access +However, users will not be prompted to sign in through SSO on each visit. GitLab will check whether a user has authenticated through SSO, and will only prompt the user to sign in via SSO if the session has expired. -To rescind access to the group: +We intend to add a similar SSO requirement for [Git and API activity](https://gitlab.com/gitlab-org/gitlab/-/issues/9152). -1. Remove the user from the identity provider or users list for the specific app. -1. Remove the user from the GitLab.com group. +When SSO enforcement is enabled for a group, users cannot share a project in the group outside the top-level group, even if the project is forked. -Even when **enforce SSO** is active, we recommend removing the user from the group. Otherwise, the user can sign in through the identity provider if they do not have an active session. +To disallow users to contribute outside of the top-level group, please see [Group Managed Accounts](group_managed_accounts.md). ## Providers -NOTE: **Note:** GitLab is unable to provide support for IdPs that are not listed here. +NOTE: **Note:** +GitLab is unable to provide support for IdPs that are not listed here. | Provider | Documentation | |----------|---------------| @@ -248,7 +109,8 @@ For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azu |--------------|----------------| | Identifier | Identifier (Entity ID) | | Assertion consumer service URL | Reply URL (Assertion Consumer Service URL) | -| Identity provider single sign on URL | Sign on URL | +| GitLab single sign-on URL | Sign on URL | +| Identity provider single sign-on URL | Login URL | | Certificate fingerprint | Thumbprint | We recommend: @@ -256,8 +118,6 @@ We recommend: - **Unique User Identifier (Name identifier)** set to `user.objectID`. - **nameid-format** set to persistent. -Set other user attributes and claims according to the [assertions table](#assertions). - ### Okta setup notes @@ -266,19 +126,17 @@ For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & S | GitLab Setting | Okta Field | |--------------|----------------| | Identifier | Audience URI | -| Assertion consumer service URL | Single sign on URL | - -Under Okta's **Single sign on URL** field, check the option **Use this for Recipient URL and Destination URL**. +| Assertion consumer service URL | Single sign-on URL | +| GitLab single sign-on URL | Login page URL (under **Application Login Page** settings) | +| Identity provider single sign-on URL | Identity Provider Single Sign-On URL | -Please note that Okta's generic SAML app does not have a **Login URL** field, where the **Identity provider single sign on URL** would normally go. The **Identity provider single sign on URL** may be required the first time a user is logging in if they are having any difficulties. +Under Okta's **Single sign-on URL** field, check the option **Use this for Recipient URL and Destination URL**. We recommend: - **Application username** (NameID) set to **Custom** `user.getInternalProperty("id")`. - **Name ID Format** set to **Persistent**. -Set attribute statements according to the [assertions table](#assertions). - ### OneLogin setup notes The GitLab app listed in the OneLogin app catalog is for self-managed GitLab instances. @@ -290,15 +148,24 @@ For GitLab.com, use a generic SAML Test Connector such as the SAML Test Connecto | Assertion consumer service URL | Recipient | | Assertion consumer service URL | ACS (Consumer) URL | | Assertion consumer service URL (escaped version) | ACS (Consumer) URL Validator | -| GitLab single sign on URL | Login URL | +| GitLab single sign-on URL | Login URL | +| Identity provider single sign-on URL | SAML 2.0 Endpoint | Recommended `NameID` value: `OneLogin ID`. -Set parameters according to the [assertions table](#assertions). +### Additional providers and setup options + +The SAML standard means that a wide range of identity providers will work with GitLab. Unfortunately we have not verified connections with all SAML providers. +For more information, see our [discussion on providers](#providers). -### Additional setup options +Your identity provider may have relevant documentation. It may be generic SAML documentation, or specifically targeted for GitLab. Examples: + +- [Auth0](https://auth0.com/docs/protocols/saml/saml-idp-generic) +- [G Suite](https://support.google.com/a/answer/6087519?hl=en) +- [JumpCloud](https://support.jumpcloud.com/support/s/article/single-sign-on-sso-with-gitlab-2019-08-21-10-36-47) +- [PingOne by Ping Identity](https://docs.pingidentity.com/bundle/pingone/page/xsh1564020480660-1.html) -GitLab [isn't limited to the SAML providers listed above](#my-identity-provider-isnt-listed) but your Identity Provider may require additional configuration, such as the following: +Your Identity Provider may require additional configuration, such as the following: | Field | Value | Notes | |-------|-------|-------| @@ -319,24 +186,48 @@ GitLab [isn't limited to the SAML providers listed above](#my-identity-provider- If the information you need isn't listed above you may wish to check our [troubleshooting docs below](#i-need-additional-information-to-configure-my-identity-provider). -## Linking SAML to your existing GitLab.com account +## User access and management + +Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, please see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup). + +When a user tries to sign in with Group SSO, they will need an account that's configured with one of the following: + +- [SCIM](scim_setup.md). +- [Group-managed accounts](group_managed_accounts.md). +- A GitLab.com account. + +### Linking SAML to your existing GitLab.com account To link SAML to your existing GitLab.com account: 1. Sign in to your GitLab.com account. -1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. -1. Visit the SSO URL and click **Authorize**. +1. Locate and visit the **GitLab single sign-on URL** for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. If the sign-in URL is configured, users can connect to the GitLab app from the Identity Provider. +1. Click **Authorize**. 1. Enter your credentials on the Identity Provider if prompted. 1. You will be redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. -## Signing in to GitLab.com with SAML +On subsequent visits, you should be able to go [sign in to GitLab.com with SAML](#signing-in-to-gitlabcom-with-saml) or by visiting links directly. If the **enforce SSO** option is turned on, you will be redirected to sign in through the identity provider. -1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on a group's **Settings > SAML SSO** page. If configured, it might also be possible to sign in to GitLab starting from your Identity Provider. -1. Visit the SSO URL and click the **Sign in with Single Sign-On** button. -1. Enter your credentials on the Identity Provider if prompted. +### Signing in to GitLab.com with SAML + +1. Sign in to your identity provider. +1. From the list of apps, click on the "GitLab.com" app (The name is set by the administrator of the identity provider). 1. You will be signed in to GitLab.com and redirected to the group. -## Unlinking accounts +### Role + +The first time you sign in, GitLab adds you to the top-level parent group with the Guest role. Existing members with appropriate privileges can promote that new user. + +If a user is already a member of the group, linking the SAML identity does not change their role. + +### Blocking access + +To rescind access to the group, perform the following steps, in order: + +1. Remove the user from the user datastore on the identity provider or the list of users on the specific app. +1. Remove the user from the GitLab.com group. + +### Unlinking accounts Users can unlink SAML for a group from their profile page. This can be helpful if: @@ -359,7 +250,7 @@ For example, to unlink the `MyOrg` account, the following **Disconnect** button | Issuer | How GitLab identifies itself to the identity provider. Also known as a "Relying party trust identifier". | | Certificate fingerprint | Used to confirm that communications over SAML are secure by checking that the server is signing communications with the correct certificate. Also known as a certificate thumbprint. | -## Configuring on a self-managed GitLab instance +## Configuring on a self-managed GitLab instance **(PREMIUM ONLY)** For self-managed GitLab instances we strongly recommend using the [instance-wide SAML OmniAuth Provider](../../../integration/saml.md) instead. @@ -377,7 +268,7 @@ Group SAML on a self-managed instance is limited when compared to the recommende [instance-wide SAML](../../../integration/saml.md). The recommended solution allows you to take advantage of: - [LDAP compatibility](../../../administration/auth/ldap/index.md). -- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap) +- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap). - [Required groups](../../../integration/saml.md#required-groups-starter-only). - [Admin groups](../../../integration/saml.md#admin-groups-starter-only). - [Auditor groups](../../../integration/saml.md#auditor-groups-starter-only). @@ -449,7 +340,6 @@ Here are possible causes and solutions: | Cause | Solution | |------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | You've tried to link multiple SAML identities to the same user, for a given Identity Provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](#unlinking-accounts) from this GitLab account before attempting to sign in again. | -| The Identity Provider might be returning an inconsistent [NameID](#nameid). | Ask an admin of your Identity Provider to use the [SCIM API](../../../api/scim.md) to update your `extern_uid` to match the current **NameID**. | ### Message: "SAML authentication failed: Email has already been taken" @@ -463,6 +353,16 @@ Getting both of these errors at the same time suggests the NameID capitalization This can be prevented by configuring the [NameID](#nameid) to return a consistent value. Fixing this for an individual user involves [unlinking SAML in the GitLab account](#unlinking-accounts), although this will cause group membership and Todos to be lost. +### Message: "Request to link SAML account must be authorized" + +Ensure that the user who is trying to link their GitLab account has been added as a user within the identity provider's SAML app. + +### Stuck in a login "loop" + +Ensure that the **GitLab single sign-on URL** has been configured as "Login URL" (or similarly named field) in the identity provider's SAML app. + +Alternatively, when users need to [link SAML to their existing GitLab.com account](#linking-saml-to-your-existing-gitlabcom-account), provide the **GitLab single sign-on URL** and instruct users not to use the SAML app on first sign in. + ### The NameID has changed | Cause | Solution | @@ -473,17 +373,6 @@ This can be prevented by configuring the [NameID](#nameid) to return a consisten Users will need to [unlink the current SAML identity](#unlinking-accounts) and [link their identity](#user-access-and-management) to the new SAML app. -### My identity provider isn't listed - -Not a problem, the SAML standard means that a wide range of identity providers will work with GitLab. Unfortunately we aren't familiar with all of them so can only offer support configuring the [listed providers](#providers). - -Your identity provider may also have relevant documentation. It may be generic SAML documentation, or specifically targeted for GitLab. Examples: - -- [Auth0](https://auth0.com/docs/protocols/saml/saml-idp-generic) -- [G Suite](https://support.google.com/a/answer/6087519?hl=en) -- [JumpCloud](https://support.jumpcloud.com/support/s/article/single-sign-on-sso-with-gitlab-2019-08-21-10-36-47) -- [PingOne by Ping Identity](https://docs.pingidentity.com/bundle/pingone/page/xsh1564020480660-1.html) - ### I need additional information to configure my identity provider Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index a891962b38e..13e9d623e2c 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -92,7 +92,8 @@ You can then test the connection by clicking on **Test Connection**. If the conn 1. Save your changes. For reference, you can view [an example configuration in the troubleshooting reference](../../../administration/troubleshooting/group_saml_scim.md#azure-active-directory). - NOTE: **Note:** If you used a unique identifier **other than** `objectId`, be sure to map it to `externalId`. + NOTE: **Note:** + If you used a unique identifier **other than** `objectId`, be sure to map it to `externalId`. 1. Below the mapping list click on **Show advanced options > Edit attribute list for AppName**. @@ -127,7 +128,8 @@ Before proceeding, be sure to complete the [GitLab configuration](#gitlab-config 1. If you see an **Admin** button in the top right, click the button. This will ensure you are in the Admin area. - TIP: **Tip:** If you're using the Developer Console, click **Developer Console** in the top + TIP: **Tip:** + If you're using the Developer Console, click **Developer Console** in the top bar and select **Classic UI**. Otherwise, you may not see the buttons described in the following steps: @@ -163,7 +165,7 @@ As long as [Group SAML](index.md) has been configured, prior to turning on sync, - By following these steps: 1. Sign in to GitLab.com if needed. - 1. Click on the GitLab app in the identity provider's dashboard or visit the **GitLab single sign on URL**. + 1. Click on the GitLab app in the identity provider's dashboard or visit the **GitLab single sign-on URL**. 1. Click on the **Authorize** button. New users and existing users on subsequent visits can access the group through the identify provider's dashboard or by visiting links directly. @@ -175,7 +177,7 @@ For role information, please see the [Group SAML page](index.md#user-access-and- To rescind access to the group, we recommend removing the user from the identity provider or users list for the specific app. -Upon the next sync, the user will be deprovisioned, which means that the user will be removed from the group. The user account will not be deleted unless using [group managed accounts](index.md#group-managed-accounts). +Upon the next sync, the user will be deprovisioned, which means that the user will be removed from the group. The user account will not be deleted unless using [group managed accounts](group_managed_accounts.md). ## Troubleshooting diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 842e2082be4..a370c9cf136 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -19,10 +19,13 @@ By using subgroups you can do the following: - **Make it easier to manage people and control visibility.** Give people different [permissions](../../permissions.md#group-members-permissions) depending on their group [membership](#membership). +For more information on allowed permissions in groups and projects, see +[visibility levels](../../../development/permissions.md#general-permissions). + ## Overview A group can have many subgroups inside it, and at the same time a group can have -only 1 parent group. It resembles a directory behavior or a nested items list: +only one immediate parent group. It resembles a directory behavior or a nested items list: - Group 1 - Group 1.1 @@ -86,7 +89,7 @@ of words that are not allowed to be used as group names see the [reserved names](../../reserved_names.md). Users can always create subgroups if they are explicitly added as an Owner (or -Maintainer, if that setting is enabled) to a parent group, even if group +Maintainer, if that setting is enabled) to an immediate parent group, even if group creation is disabled by an administrator in their settings. To create a subgroup: @@ -96,9 +99,9 @@ To create a subgroup: ![Subgroups page](img/create_subgroup_button.png) -1. Create a new group like you would normally do. Notice that the parent group +1. Create a new group like you would normally do. Notice that the immediate parent group namespace is fixed under **Group path**. The visibility level can differ from - the parent group. + the immediate parent group. ![Subgroups page](img/create_new_group.png) @@ -110,12 +113,13 @@ Follow the same process to create any subsequent groups. ## Membership When you add a member to a subgroup, they inherit the membership and permission -level from the parent group. This model allows access to nested groups if you +level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. -Jobs for pipelines in subgroups can use [Runners](../../../ci/runners/README.md) registered to the parent group. This means secrets configured for the parent group are available to subgroup jobs. +Jobs for pipelines in subgroups can use [Runners](../../../ci/runners/README.md) registered to the parent group(s). +This means secrets configured for the parent group are available to subgroup jobs. -In addition, maintainers of projects that belong to subgroups can see the details of Runners registered to parent groups. +In addition, maintainers of projects that belong to subgroups can see the details of Runners registered to parent group(s). The group permissions for a member can be changed only by Owners, and only on the **Members** page of the group the member was added. diff --git a/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png b/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png new file mode 100644 index 00000000000..9277b013676 Binary files /dev/null and b/doc/user/incident_management/img/pagerduty_incidents_integration_13_2.png differ diff --git a/doc/user/incident_management/index.md b/doc/user/incident_management/index.md index 48805bda909..996f423e770 100644 --- a/doc/user/incident_management/index.md +++ b/doc/user/incident_management/index.md @@ -25,7 +25,7 @@ to create issues when alerts are triggered: checkbox to create an issue based on your own [issue templates](../project/description_templates.md#creating-issue-templates). For more information, see - [Taking Action on Incidents](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) **(ULTIMATE)**. + [Trigger actions from alerts](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) **(ULTIMATE)**. 1. To create issues from alerts, select the template in the **Issue Template** select box. 1. To send [separate email notifications](#notify-developers-of-alerts) to users @@ -34,9 +34,9 @@ to create issues when alerts are triggered: 1. Click **Save changes**. Appropriately configured alerts include an -[embedded chart](../project/integrations/prometheus.md#embedding-metrics-based-on-alerts-in-incident-issues) +[embedded chart](../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues) for the query corresponding to the alert. You can also configure GitLab to -[close issues](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) +[close issues](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) when you receive notification that the alert is resolved. ### Notify developers of alerts @@ -49,12 +49,35 @@ These emails contain details of the alert, and a link for more information. To send separate email notifications to users with [Developer permissions](../permissions.md), see [Configure incidents](#configure-incidents-ultimate). +## Configure PagerDuty integration + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/119018) in GitLab 13.2. + +You can set up a webhook with PagerDuty to automatically create a GitLab issue +for each PagerDuty incident. This configuration requires you to make changes +in both PagerDuty and GitLab: + +1. Sign in as a user with Maintainer [permissions](../permissions.md). +1. Navigate to **{settings}** **Settings > Operations > Incidents** and expand **Incidents**. +1. Select the **PagerDuty integration** tab: + + ![PagerDuty incidents integration](img/pagerduty_incidents_integration_13_2.png) + +1. Activate the integration, and save the changes in GitLab. +1. Copy the value of **Webhook URL**, as you'll need it in a later step. +1. Follow the steps described in the + [PagerDuty documentation](https://support.pagerduty.com/docs/webhooks) + to add the webhook URL to a PagerDuty webhook integration. + +To confirm the integration is successful, trigger a test incident from PagerDuty to +confirm that a GitLab issue is created from the incident. + ## Configure Prometheus alerts You can set up Prometheus alerts in: -- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics) installations. -- [Self-managed Prometheus](../project/integrations/prometheus.md#external-prometheus-instances) installations. +- [GitLab-managed Prometheus](../../operations/metrics/alerts.md) installations. +- [Self-managed Prometheus](../../operations/metrics/alerts.md#external-prometheus-instances) installations. Prometheus alerts are created by the special Alert Bot user. You can't remove this user, but it does not count toward your license limit. @@ -71,11 +94,11 @@ You can embed metrics anywhere [GitLab Markdown](../markdown.md) is used, such a comments on issues, and merge requests. Embedding metrics helps you share them when discussing incidents or performance issues. You can output the dashboard directly into any issue, merge request, epic, or any other Markdown text field in GitLab -by [copying and pasting the link to the metrics dashboard](../project/integrations/prometheus.md#embedding-gitlab-managed-kubernetes-metrics). +by [copying and pasting the link to the metrics dashboard](../../operations/metrics/embed.md#embedding-gitlab-managed-kubernetes-metrics). You can embed both -[GitLab-hosted metrics](../project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) and -[Grafana metrics](../project/integrations/prometheus.md#embedding-grafana-charts) +[GitLab-hosted metrics](../../operations/metrics/embed.md) and +[Grafana metrics](../../operations/metrics/embed_grafana.md) in incidents and issue templates. ### Context menu @@ -86,7 +109,7 @@ above the upper right corner of the panel. The options are: - [View logs](#view-logs-from-metrics-panel). - **Download CSV** - Data from embedded charts can be - [downloaded as CSV](../project/integrations/prometheus.md#downloading-data-as-csv). + [downloaded as CSV](../../operations/metrics/dashboards/index.md#downloading-data-as-csv). #### View logs from metrics panel @@ -94,7 +117,7 @@ above the upper right corner of the panel. The options are: > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) to [GitLab Core](https://about.gitlab.com/pricing/) 12.9. Viewing logs from a metrics panel can be useful if you're triaging an application -incident and need to [explore logs](../project/integrations/prometheus.md#view-logs-ultimate) +incident and need to [explore logs](../../operations/metrics/dashboards/index.md#view-logs-ultimate) from across your application. These logs help you understand what is affecting your application's performance and resolve any problems. diff --git a/doc/user/index.md b/doc/user/index.md index cc521c2a767..f50b712e2c3 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -46,10 +46,10 @@ GitLab is a Git-based platform that integrates a great number of essential tools - Deploying personal and professional static websites with [GitLab Pages](project/pages/index.md). - Integrating with Docker by using [GitLab Container Registry](packages/container_registry/index.md). - Tracking the development lifecycle by using [GitLab Value Stream Analytics](project/cycle_analytics.md). +- Provide support with [Service Desk](project/service_desk.md). With GitLab Enterprise Edition, you can also: -- Provide support with [Service Desk](project/service_desk.md). - Improve collaboration with: - [Merge Request Approvals](project/merge_requests/merge_request_approvals.md). **(STARTER)** - [Multiple Assignees for Issues](project/issues/multiple_assignees_for_issues.md). **(STARTER)** @@ -148,7 +148,7 @@ requests you're assigned to. [Snippets](snippets.md) are code blocks that you want to store in GitLab, from which you have quick access to. You can also gather feedback on them through -[Discussions](#Discussions). +[Discussions](#discussions). ## Keyboard shortcuts diff --git a/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png b/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png new file mode 100644 index 00000000000..564835a5dbe Binary files /dev/null and b/doc/user/infrastructure/img/terraform_plan_widget_v13_2.png differ diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index c17d1831b50..e24e669d994 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -6,8 +6,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Infrastructure as code with Terraform and GitLab +## Motivation + +The Terraform integration features within GitLab enable your GitOps / Infrastructure-as-Code (IaC) +workflows to tie into GitLab's authentication and authorization. These features focus on +lowering the barrier to entry for teams to adopt Terraform, collaborate effectively within +GitLab, and support Terraform best practices. + ## GitLab managed Terraform State +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. + [Terraform remote backends](https://www.terraform.io/docs/backends/index.html) enable you to store the state file in a remote, shared store. GitLab uses the [Terraform HTTP backend](https://www.terraform.io/docs/backends/types/http.html) @@ -27,6 +36,14 @@ To get started with a GitLab-managed Terraform State, there are two different op - [Use a local machine](#get-started-using-local-development). - [Use GitLab CI](#get-started-using-gitlab-ci). +## Permissions for using Terraform + +In GitLab version 13.1, [Maintainer access](../permissions.md) was required to use a +GitLab managed Terraform state backend. In GitLab versions 13.2 and greater, +[Maintainer access](../permissions.md) is required to lock, unlock and write to the state +(using `terraform apply`), while [Developer access](../permissions.md) is required to read +the state (using `terraform plan -lock=false`). + ## Get started using local development If you plan to only run `terraform plan` and `terraform apply` commands from your @@ -45,8 +62,7 @@ local machine, this is a simple way to get started: ``` 1. Create a [Personal Access Token](../profile/personal_access_tokens.md) with - the `api` scope. The Terraform backend is restricted to users with - [Maintainer access](../permissions.md) to the repository. + the `api` scope. 1. On your local machine, run `terraform init`, passing in the following options, replacing ``, ``, `` and @@ -80,10 +96,6 @@ Next, [configure the backend](#configure-the-backend). After executing the `terraform init` command, you must configure the Terraform backend and the CI YAML file: -CAUTION: **Important:** -The Terraform backend is restricted to users with [Maintainer access](../permissions.md) -to the repository. - 1. In your Terraform project, define the [HTTP backend](https://www.terraform.io/docs/backends/types/http.html) by adding the following code block in a `.tf` file (such as `backend.tf`) to define the remote backend: @@ -95,64 +107,75 @@ to the repository. } ``` -1. In the root directory of your project repository, configure a `.gitlab-ci.yaml` file. - This example uses a pre-built image: +1. In the root directory of your project repository, configure a + `.gitlab-ci.yaml` file. This example uses a pre-built image which includes a + `gitlab-terraform` helper. For supported Terraform versions, see the [GitLab + Terraform Images project](https://gitlab.com/gitlab-org/terraform-images). ```yaml - image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest ``` -1. In the `.gitlab-ci.yaml` file, define some environment variables to ease development. In this - example, `GITLAB_TF_ADDRESS` is the URL of the GitLab instance where this pipeline - runs, and `TF_ROOT` is the directory where the Terraform commands must be executed: +1. In the `.gitlab-ci.yaml` file, define some environment variables to ease + development. In this example, `TF_ROOT` is the directory where the Terraform + commands must be executed, `TF_ADDRESS` is the URL to the state on the GitLab + instance where this pipeline runs, and the final path segment in `TF_ADDRESS` + is the name of the Terraform state. Projects may have multiple states, and + this name is arbitrary, so in this example we will set it to the name of the + project, and we will ensure that the `.terraform` directory is cached between + jobs in the pipeline using a cache key based on the state name: ```yaml variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production + TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} cache: + key: ${CI_PROJECT_NAME} paths: - - .terraform + - ${TF_ROOT}/.terraform ``` -1. In a `before_script`, pass a `terraform init` call containing configuration parameters - corresponding to variables required by the - [HTTP backend](https://www.terraform.io/docs/backends/types/http.html): +1. In a `before_script`, change to your `TF_ROOT`: ```yaml before_script: - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=gitlab-ci-token" -backend-config="password=${CI_JOB_TOKEN}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" stages: + - prepare - validate - build - - test - deploy + init: + stage: prepare + script: + - gitlab-terraform init + validate: stage: validate script: - - terraform validate + - gitlab-terraform validate plan: stage: build script: - - terraform plan - - terraform show + - gitlab-terraform plan + - gitlab-terraform plan-json + artifacts: + name: plan + paths: + - ${TF_ROOT}/plan.cache + reports: + terraform: ${TF_ROOT}/plan.json apply: stage: deploy environment: name: production script: - - terraform apply + - gitlab-terraform apply dependencies: - plan when: manual @@ -160,8 +183,9 @@ to the repository. - master ``` -1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline runs - the `terraform init`, `terraform validate`, and `terraform plan` commands. +1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline + runs the `gitlab-terraform init`, `gitlab-terraform validate`, and + `gitlab-terraform plan` commands. The output from the above `terraform` commands should be viewable in the job logs. @@ -176,15 +200,18 @@ you can expose details from `terraform plan` runs directly into a merge request enabling you to see statistics about the resources that Terraform will create, modify, or destroy. -Let's explore how to configure a GitLab Terraform Report artifact: +Let's explore how to configure a GitLab Terraform Report artifact. You can +either use a pre-built image which includes a `gitlab-terraform` helper as +above, where `gitlab-terraform plan-json` outputs the required artifact, or you +can configure this manually as follows: 1. For simplicity, let's define a few reusable variables to allow us to refer to these files multiple times: ```yaml variables: - PLAN: plan.tfplan - PLAN_JSON: tfplan.json + PLAN: plan.cache + PLAN_JSON: plan.json ``` 1. Install `jq`, a @@ -198,6 +225,18 @@ Let's explore how to configure a GitLab Terraform Report artifact: - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" ``` + NOTE: **Note:** + In distributions that use Bash (for example, Ubuntu), `alias` statements are not + expanded in non-interactive mode. If your pipelines fail with the error + `convert_report: command not found`, alias expansion can be activated explicitly + by adding a `shopt` command to your script: + + ```yaml + before_script: + - shopt -s expand_aliases + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" + ``` + 1. Define a `script` that runs `terraform plan` and `terraform show`. These commands pipe the output and convert the relevant bits into a store variable `PLAN_JSON`. This JSON is used to create a @@ -212,18 +251,18 @@ Let's explore how to configure a GitLab Terraform Report artifact: - terraform plan -out=$PLAN - terraform show --json $PLAN | convert_report > $PLAN_JSON artifacts: - name: plan - paths: - - $PLAN reports: terraform: $PLAN_JSON ``` - For a full example, see [Example `.gitlab-ci.yaml` file](#example-gitlab-ciyaml-file). + For a full example using the pre-built image, see [Example `.gitlab-ci.yaml` + file](#example-gitlab-ciyaml-file). + + For an example displaying multiple reports, see [`.gitlab-ci.yaml` multiple reports file](#multiple-terraform-plan-reports). 1. Running the pipeline displays the widget in the merge request, like this: - ![MR Terraform widget](img/terraform_plan_widget_v13_0.png) + ![Merge Request Terraform widget](img/terraform_plan_widget_v13_2.png) 1. Clicking the **View Full Log** button in the widget takes you directly to the plan output present in the pipeline logs: @@ -233,64 +272,114 @@ Let's explore how to configure a GitLab Terraform Report artifact: ### Example `.gitlab-ci.yaml` file ```yaml -image: - name: hashicorp/terraform:light - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' +image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest -# Default output file for Terraform plan variables: - GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} - PLAN: plan.tfplan - PLAN_JSON: tfplan.json - TF_ROOT: ${CI_PROJECT_DIR} + TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production + TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} cache: + key: ${CI_PROJECT_NAME} paths: - - .terraform + - ${TF_ROOT}/.terraform before_script: - - apk --no-cache add jq - - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" - cd ${TF_ROOT} - - terraform --version - - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" stages: + - prepare - validate - build - deploy +init: + stage: prepare + script: + - gitlab-terraform init + validate: stage: validate script: - - terraform validate + - gitlab-terraform validate plan: stage: build script: - - terraform plan -out=$PLAN - - terraform show --json $PLAN | convert_report > $PLAN_JSON + - gitlab-terraform plan + - gitlab-terraform plan-json artifacts: name: plan paths: - - ${TF_ROOT}/plan.tfplan + - ${TF_ROOT}/plan.cache reports: - terraform: ${TF_ROOT}/tfplan.json + terraform: ${TF_ROOT}/plan.json -# Separate apply job for manual launching Terraform as it can be destructive -# action. apply: stage: deploy environment: name: production script: - - terraform apply -input=false $PLAN + - gitlab-terraform apply dependencies: - plan when: manual only: - master +``` + +### Multiple Terraform Plan reports + +Starting with 13.2, you can display mutiple reports on the Merge Request page. The reports will also display the `artifact: name:`. See example below for a suggested setup. + +```yaml +image: + name: registry.gitlab.com/gitlab-org/gitlab-build-images:terraform + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + +cache: + paths: + - .terraform +stages: + - build + +.terraform-plan-generation: + stage: build + variables: + PLAN: plan.tfplan + JSON_PLAN_FILE: tfplan.json + before_script: + - cd ${TERRAFORM_DIRECTORY} + - terraform --version + - terraform init + - apk --no-cache add jq + script: + - terraform validate + - terraform plan -out=${PLAN} + - terraform show --json ${PLAN} | jq -r '([.resource_changes[]?.change.actions?]|flatten)|{"create":(map(select(.=="create"))|length),"update":(map(select(.=="update"))|length),"delete":(map(select(.=="delete"))|length)}' > ${JSON_PLAN_FILE} + artifacts: + reports: + terraform: ${TERRAFORM_DIRECTORY}/${JSON_PLAN_FILE} + +review_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "review/" + # Review will not include an artifact name + +staging_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "staging/" + artifacts: + name: Staging + +production_plan: + extends: .terraform-plan-generation + variables: + TERRAFORM_DIRECTORY: "production/" + artifacts: + name: Production ``` diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 0d028cfe77a..b2b3f0f925c 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -14,7 +14,8 @@ website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitl Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/markdown-guide/) for a complete Kramdown reference. -NOTE: **Note:** We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md). +NOTE: **Note:** +We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md). ## GitLab Flavored Markdown (GFM) @@ -54,7 +55,7 @@ repository that were written using some of the nuances of GitLab's RedCarpet ver of Markdown. Since CommonMark uses slightly stricter syntax, these documents might now appear a little differently since we have transitioned to CommonMark. -It's usually quite easy to fix. For example, numbered lists with nested lists may +For example, numbered lists with nested lists may render incorrectly: ```markdown @@ -63,8 +64,8 @@ render incorrectly: - milk ``` -Simply add a space to each nested item to align the `-` with the first character of -the top list item (`C` in this case): +To correct their rendering, add a space to each nested item to align the `-` with the first +character of the top list item (`C` in this case): ```markdown 1. Chocolate @@ -76,7 +77,8 @@ the top list item (`C` in this case): - dark - milk -NOTE: **Note:** We will flag any significant differences between Redcarpet and CommonMark +NOTE: **Note:** +We will flag any significant differences between Redcarpet and CommonMark Markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine @@ -92,7 +94,7 @@ to change. GitLab makes full use of the standard (CommonMark) formatting, but also includes additional functionality useful for GitLab users. -It makes use of [new Markdown features](#new-GFM-markdown-extensions), +It makes use of [new Markdown features](#new-gfm-markdown-extensions), not found in standard Markdown: - [Color "chips" written in HEX, RGB or HSL](#colors) @@ -241,7 +243,7 @@ Sometimes you want to :monkey: around a bit and add some :star2: to your :speech You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. -If you're new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. +If you're new to this, don't be :fearful:. You can join the emoji :family:. All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: ``` @@ -252,7 +254,7 @@ Sometimes you want to or warn about patches. And if someone improves your really code, send them some . People will you for that. -If you're new to this, don't be . You can easily join the emoji . All you need to do is to look up one of the supported codes. +If you're new to this, don't be . You can join the emoji . All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. @@ -261,7 +263,8 @@ when rendered within GitLab, may appear different depending on the OS and browse Most emoji are natively supported on macOS, Windows, iOS, Android, and will fall back on image-based emoji where there is no support. -NOTE: **Note:** On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) +NOTE: **Note:** +On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. Ubuntu 18.04 (like many modern Linux distributions) has this font installed by default. @@ -402,14 +405,15 @@ a^2+b^2=c^2 _Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -NOTE: **Note:** This also works for the Asciidoctor `:stem: latexmath`. For details see +NOTE: **Note:** +This also works for the Asciidoctor `:stem: latexmath`. For details see the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). ### Special GitLab references -GFM recognizes special GitLab related references. For example, you can easily reference +GFM recognizes special GitLab related references. For example, you can reference an issue, a commit, a team member, or even the whole team within a project. GFM will turn -that reference into a link so you can navigate between them easily. +that reference into a link so you can navigate between them. Additionally, GFM recognizes certain cross-project references and also has a shorthand version to reference other projects from the same namespace. @@ -502,8 +506,6 @@ Second section content. ![Preview of an auto-generated TOC in a Wiki](img/markdown_toc_preview_v12_9.png) ---- - ### Wiki-specific Markdown The following examples show how links inside wikis behave. @@ -581,7 +583,7 @@ This snippet links to `/miscellaneous.md`: ### Embedding metrics in GitLab Flavored Markdown -Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab flavored Markdown](../user/project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. +Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab flavored Markdown](../operations/metrics/embed.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. ## Standard Markdown and extensions in GitLab @@ -591,7 +593,7 @@ If a functionality is extended, the new option will be listed as a sub-section. ### Blockquotes -Blockquotes are an easy way to highlight information, such as a side-note. It's generated +Blockquotes are useful to highlight information, such as a side-note. It's generated by starting the lines of the blockquote with `>`: ```markdown @@ -637,9 +639,9 @@ you can quote that without having to manually prepend `>` to every line! ### Code spans and blocks -You can easily highlight anything that should be viewed as code and not simple text. +You can highlight anything that should be viewed as code and not simple text. -Simple inline code is easily highlighted with single backticks `` ` ``: +Simple inline code is highlighted with single backticks `` ` ``: ```markdown Inline `code` has `back-ticks around` it. @@ -781,7 +783,8 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ -NOTE: **Note:** Strikethrough is not part of the core Markdown standard, but is part of GFM. +NOTE: **Note:** +Strikethrough is not part of the core Markdown standard, but is part of GFM. #### Multiple underscores in words and mid-word emphasis @@ -1150,7 +1153,7 @@ A line break will be inserted (a new paragraph will start) if the previous text ended with two newlines, like when you hit Enter twice in a row. If you only use one newline (hit Enter once), the next sentence will be part of the same paragraph. This is useful if you want to keep long lines from wrapping, and keep -them easily editable: +them editable: ```markdown Here's a line for us to start with. @@ -1248,7 +1251,8 @@ Do not change to reference style links. Some text to show that the reference links can follow later. -NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki +NOTE: **Note:** +Relative links do not allow the referencing of project files in a wiki page, or a wiki page in a project file. The reason for this is that a wiki is always in a separate Git repository in GitLab. For example, `[I'm a reference-style link](style)` will point the link to `wikis/style` only when the link is inside of a wiki Markdown file. @@ -1275,7 +1279,7 @@ GFM will auto-link almost any URL you put into your text: ### Lists -Ordered and unordered lists can be easily created. +Ordered and unordered lists can be created. For an ordered list, add the number you want the list to start with, like `1.`, followed by a space, at the start of each line for ordered lists. diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 8a7c70ec74d..542efba1fae 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Composer Repository **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15886) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15886) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. With the GitLab Composer Repository, every project can have its own space to store [Composer](https://getcomposer.org/) packages. @@ -19,7 +19,7 @@ This option is available only if your GitLab administrator has After the Composer Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 020028dfc10..41b420ce7f6 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -22,7 +22,7 @@ This option is available only if your GitLab administrator has After the Conan Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -85,7 +85,7 @@ Next, you will create a package for that recipe by running `conan create` provid conan create . my-org+my-group+my-project/beta ``` -NOTE: **Note** +NOTE: **Note:** Current [naming restrictions](#package-recipe-naming-convention) require you to name the `user` value as the `+` separated path of your project on GitLab. The example above would create a package belonging to this project: `https://gitlab.com/my-org/my-group/my-project` with a channel of `beta`. @@ -129,12 +129,12 @@ Once you have a personal access token and have [set your Conan remote](#adding-t conan user -r gitlab -p ``` -Note: **Note** +NOTE: **Note:** If you named your remote something other than `gitlab`, your remote name should be used in this command instead of `gitlab`. From now on, when you run commands using `--remote=gitlab`, your username and password will automatically be included in the requests. -Note: **Note** +NOTE: **Note:** The personal access token is not stored locally at any moment. Conan uses JWT, so when you run this command, Conan will request an expirable token from GitLab using your token. The JWT does expire on a regular basis, so you will need to re-enter your personal access token when that happens. Alternatively, you could explicitly include your credentials in any given command. @@ -152,7 +152,7 @@ If you'd like Conan to always use GitLab as the registry for your package, you c conan remote add_ref Hello/0.1@my-group+my-project/beta gitlab ``` -NOTE: **Note** +NOTE: **Note:** The package recipe does include the version, so setting the default remote for `Hello/0.1@user/channel` will not work for `Hello/0.2@user/channel`. This functionality is best suited for when you want to consume or install packages from the GitLab registry without having to specify a remote. diff --git a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png deleted file mode 100644 index 93c9e00a8f5..00000000000 Binary files a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png and /dev/null differ diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index eb1933de62a..429d29b7677 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -71,7 +71,7 @@ This view will: - Allow you to [delete](#delete-images-from-within-gitlab) one or more image repository. - Allow you to navigate to the image repository details page. - Show a **Quick start** dropdown with the most common commands to log in, build and push -- Optionally, a banner will be visible if the [expiration policy](#expiration-policy) is enabled for this project. +- Optionally, a banner will be visible if the [cleanup policy](#cleanup-policy) is enabled for this project. ### Control Container Registry for your group @@ -248,10 +248,10 @@ should look similar to this: ```yaml build: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY/group/project/image:latest . @@ -262,10 +262,10 @@ You can also make use of [other variables](../../../ci/variables/README.md) to a ```yaml build: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -288,9 +288,9 @@ when needed. Changes to `master` also get tagged as `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:19.03.11 +image: docker:19.03.12 services: - - docker:19.03.11-dind + - docker:19.03.12-dind stages: - build @@ -363,9 +363,9 @@ Below is an example of what your `.gitlab-ci.yml` should look like: ```yaml build: - image: $CI_REGISTRY/group/project/docker:19.03.11 + image: $CI_REGISTRY/group/project/docker:19.03.12 services: - - name: $CI_REGISTRY/group/project/docker:19.03.11-dind + - name: $CI_REGISTRY/group/project/docker:19.03.12-dind alias: docker stage: build script: @@ -373,7 +373,7 @@ Below is an example of what your `.gitlab-ci.yml` should look like: - docker run my-docker-image /script/to/run/tests ``` -If you forget to set the service alias, the `docker:19.03.11` image won't find the +If you forget to set the service alias, the `docker:19.03.12` image won't find the `dind` service, and an error like the following will be thrown: ```plaintext @@ -443,10 +443,10 @@ stages: - clean build_image: - image: docker:19.03.11 + image: docker:19.03.12 stage: build services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -459,10 +459,10 @@ build_image: - master delete_image: - image: docker:19.03.11 + image: docker:19.03.12 stage: clean services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: IMAGE_TAG: $CI_PROJECT_PATH:$CI_COMMIT_REF_SLUG REG_SHA256: ade837fc5224acd8c34732bf54a94f579b47851cc6a7fd5899a98386b782e228 @@ -486,25 +486,33 @@ You can download the latest `reg` release from the code example by changing the `REG_SHA256` and `REG_VERSION` variables defined in the `delete_image` job. -### Delete images using an expiration policy +### Delete images by using a cleanup policy -You can create a per-project [expiration policy](#expiration-policy) to ensure -older tags and images are regularly removed from the Container Registry. +You can create a per-project [cleanup policy](#cleanup-policy) to ensure older tags and images are regularly removed from the +Container Registry. -## Expiration policy +## Cleanup policy -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15398) in GitLab 12.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15398) in GitLab 12.8. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/218737) from "expiration policy" to "cleanup policy" in GitLab 13.2. + +For a specific project, if you want to remove tags you no longer need, +you can create a cleanup policy. When the policy is applied, tags matching the regex pattern are removed. +The underlying layers and images remain. + +To delete the underlying layers and images no longer associated with any tags, Instance Administrators can use +[garbage collection](../../../administration/packages/container_registry.md#removing-unused-layers-not-referenced-by-manifests) with the `-m` switch. NOTE: **Note:** -For GitLab.com, expiration policies are not available for projects created before GitLab 12.8. -For self-managed instances, expiration policies may be enabled by an admin in the -[CI/CD Package Registry settings](./../../admin_area/settings/index.md#cicd). +For GitLab.com, cleanup policies are not available for projects created +before this feature was deployed to production (February 2020). +Support for pre-existing projects on GitLab.com +[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/196124). +For self-managed instances, cleanup policies may be enabled by an admin in the +[GitLab application settings](../../../api/settings.md#change-application-settings) by setting `container_expiration_policies_enable_historic_entries` to true. Note the inherent [risks involved](./index.md#use-with-external-container-registries). -It is possible to create a per-project expiration policy, so that you can make sure that -older tags and images are regularly removed from the Container Registry. - -The expiration policy algorithm starts by collecting all the tags for a given repository in a list, +The cleanup policy algorithm starts by collecting all the tags for a given repository in a list, then goes through a process of excluding tags from it until only the ones to be deleted remain: 1. Collect all the tags for a given repository in a list. @@ -513,43 +521,41 @@ then goes through a process of excluding tags from it until only the ones to be 1. Excludes any tags that do not have a manifest (not part of the options). 1. Orders the remaining tags by `created_date`. 1. Excludes from the list the N tags based on the `keep_n` value (Number of tags to retain). -1. Excludes from the list the tags more recent than the `older_than` value (Expiration interval). +1. Excludes from the list the tags more recent than the `older_than` value (Cleanup interval). 1. Excludes from the list any tags matching the `name_regex_keep` value (Images to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. -### Managing project expiration policy through the UI - -To manage project expiration policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag expiration policy**. +### Managing project cleanup policy through the UI -![Expiration Policy App](img/expiration_policy_app_v13_0.png) +To manage project cleanup policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag cleanup policy**. The UI allows you to configure the following: -- **Expiration policy:** enable or disable the expiration policy. -- **Expiration interval:** how long tags are exempt from being deleted. -- **Expiration schedule:** how often the cron job checking the tags should run. +- **Cleanup policy:** enable or disable the cleanup policy. +- **Cleanup interval:** how long tags are exempt from being deleted. +- **Cleanup schedule:** how often the cron job checking the tags should run. - **Number of tags to retain:** how many tags to _always_ keep for each image. -- **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be expired. To qualify all tags for expiration, use the default value of `.*`. +- **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be cleaned up. To qualify all tags for cleanup, use the default value of `.*`. - **Docker tags with names matching this regex pattern will be preserved:** the regex used to determine what tags should be preserved. To preserve all tags, use the default value of `.*`. -#### Troubleshooting expiration policies +#### Troubleshooting cleanup policies If you see the following message: -"Something went wrong while updating the expiration policy." +"Something went wrong while updating the cleanup policy." Check the regex patterns to ensure they are valid. You can use [Rubular](https://rubular.com/) to check your regex. View some common [regex pattern examples](#regex-pattern-examples). -### Managing project expiration policy through the API +### Managing project cleanup policy through the API -You can set, update, and disable the expiration policies using the GitLab API. +You can set, update, and disable the cleanup policies using the GitLab API. Examples: -- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: +- Select all tags, keep at least 1 tag per image, clean up any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: ```shell curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: " --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' 'https://gitlab.example.com/api/v4/projects/2' @@ -560,15 +566,15 @@ See the API documentation for further details: [Edit project](../../../api/proje ### Use with external container registries When using an [external container registry](./../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), -running an expiration policy on a project may have some performance risks. If a project is going to run +running a cleanup policy on a project may have some performance risks. If a project is going to run a policy that will remove large quantities of tags (in the thousands), the GitLab background jobs that -run the policy may get backed up or fail completely. It is recommended you only enable container expiration +run the policy may get backed up or fail completely. It is recommended you only enable container cleanup policies for projects that were created before GitLab 12.8 if you are confident the amount of tags being cleaned up will be minimal. ### Regex pattern examples -Expiration policies use regex patterns to determine which tags should be preserved or removed, both in the UI and the API. +Cleanup policies use regex patterns to determine which tags should be preserved or removed, both in the UI and the API. Here are examples of regex patterns you may want to use: @@ -596,6 +602,15 @@ Here are examples of regex patterns you may want to use: (?:v.+|master|release) ``` +## Use the Container Registry to store Helm Charts + +With the launch of [Helm v3](https://helm.sh/docs/topics/registries/), +you can use the Container Registry to store Helm Charts. However, due to the way metadata is passed +and stored by Docker, it is not possible for GitLab to parse this data and meet performance standards. +[This epic](https://gitlab.com/groups/gitlab-org/-/epics/2313) updates the architecture of the Container Registry to support Helm Charts. + +You can read more about the above challenges [here](https://gitlab.com/gitlab-org/gitlab/-/issues/38047#note_298842890). + ## Limitations - Moving or renaming existing Container Registry repositories is not supported @@ -603,7 +618,7 @@ once you have pushed images, because the images are signed, and the signature includes the repository name. To move or rename a repository with a Container Registry, you will have to delete all existing images. - Prior to GitLab 12.10, any tags that use the same image ID as the `latest` tag -will not be deleted by the expiration policy. +will not be deleted by the cleanup policy. ## Troubleshooting the GitLab Container Registry diff --git a/doc/user/packages/img/group_packages_list_v13_0.png b/doc/user/packages/img/group_packages_list_v13_0.png deleted file mode 100644 index 8cf3fd1a131..00000000000 Binary files a/doc/user/packages/img/group_packages_list_v13_0.png and /dev/null differ diff --git a/doc/user/packages/img/package_detail_v13_0.png b/doc/user/packages/img/package_detail_v13_0.png deleted file mode 100644 index dcfbc0a4787..00000000000 Binary files a/doc/user/packages/img/package_detail_v13_0.png and /dev/null differ diff --git a/doc/user/packages/img/project_packages_list_v13_0.png b/doc/user/packages/img/project_packages_list_v13_0.png deleted file mode 100644 index 468a6fe6467..00000000000 Binary files a/doc/user/packages/img/project_packages_list_v13_0.png and /dev/null differ diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 6e59a87ae36..ab9cdc204f8 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -6,11 +6,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Package Registry -GitLab Packages allows organizations to utilize GitLab as a private repository -for a variety of common package managers. Users are able to build and publish +With the GitLab Package Registry, you can use GitLab as a private or public repository +for a variety of common package managers. You can build and publish packages, which can be easily consumed as a dependency in downstream projects. -The Packages feature allows GitLab to act as a repository for the following: +GitLab acts as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | @@ -22,87 +22,88 @@ The Packages feature allows GitLab to act as a repository for the following: | [NuGet Repository](nuget_repository/index.md) **(PREMIUM)** | The GitLab NuGet Repository will enable every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.8+ | | [PyPi Repository](pypi_repository/index.md) **(PREMIUM)** | The GitLab PyPi Repository will enable every project in GitLab to have its own space to store [PyPi](https://pypi.org/) packages. | 12.10+ | | [Go Proxy](go_proxy/index.md) **(PREMIUM)** | The Go proxy for GitLab enables every project in GitLab to be fetched with the [Go proxy protocol](https://proxy.golang.org/). | 13.1+ | -| [Composer Repository](composer_repository/index.md) **(PREMIUM)** | The GitLab Composer Repository will enable every project in GitLab to have its own space to store [Composer](https://getcomposer.org/) packages. | 13.1+ | +| [Composer Repository](composer_repository/index.md) **(PREMIUM)** | The GitLab Composer Repository will enable every project in GitLab to have its own space to store [Composer](https://getcomposer.org/) packages. | 13.2+ | -## Enable the Package Registry for your project +## View packages -If you cannot find the **{package}** **Packages & Registries > Package -Registry** entry under your project's sidebar, ensure that: +You can view packages for your project or group. -1. The GitLab Package Registry has been enabled by your administrator (following - [this documentation](../../administration/packages/index.md)); and -1. The Package Registry has been enabled for your project. +1. Go to the project or group. +1. Go to **{package}** **Packages & Registries > Package Registry**. -Once an administrator has enabled the GitLab Package Registry for your GitLab -instance, to enable Package Registry for your project: +You can search, sort, and filter packages on this page. -1. Go to your project's **Settings > General** page. -1. Expand the **Visibility, project features, permissions** section and enable the -**Packages** feature on your project. -1. Press **Save changes** for the changes to take effect. You should now be able to -see the **Packages & Registries > Package Registry** link in the sidebar. +For information on how to create and upload a package, view the GitLab documentation for your package type. -### View Packages for your project +## Use GitLab CI/CD to build packages -Navigating to your project's **{package}** **Packages & Registries > Package Registry** will show a list -of all packages that have been added to your project. +You can use [GitLab CI/CD](./../../ci/README.md) to build packages. +For Maven and NPM packages, and Composer dependencies, you can +authenticate with GitLab by using the `CI_JOB_TOKEN`. -![Project Packages list](img/project_packages_list_v13_0.png) +CI/CD templates, which you can use to get started, are in [this repo](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). -On this page, you can: +Learn more about [using CI/CD to build Maven packages](maven_repository/index.md#creating-maven-packages-with-gitlab-cicd) +and [NPM packages](npm_registry/index.md#publishing-a-package-with-cicd). -- View all the packages that have been uploaded to the project. -- Sort the packages list by created date, version or name. -- Filter the list by package name. -- Change tabs to display packages of a certain type. -- Remove a package (if you have suitable [permissions](../permissions.md)). -- Navigate to specific package detail page. +If you use CI/CD to build a package, extended activity +information is displayed when you view the package details: -### View Packages for your group +![Package CI/CD activity](img/package_activity_v12_10.png) -You can view all packages belonging to a group by navigating to **{package}** -**Packages & Registries > Package Registry** from the group sidebar. +You can view which pipeline published the package, as well as the commit and +user who triggered it. -![Group Packages list](img/group_packages_list_v13_0.png) +## Download a package -On this page, you can: +To download a package: -- View all the packages that have been uploaded to each of the groups projects. -- Sort the packages list by created date, version, name or project. -- Filter the list by package name. -- Change tabs to display packages of a certain type. -- Navigate to specific package detail page. +1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Click the name of the package you want to download. +1. In the **Activity** section, click the name of the package you want to download. -### View additional package information +## Delete a package -Additional package information can be viewed by browsing to the package details -page from the either the project or group list. +You cannot edit a package after you publish it in the Package Registry. Instead, you +must delete and recreate it. -![Package detail](img/package_detail_v13_0.png) +- You cannot delete packages from the group view. You must delete them from the project view instead. + See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/227714) for details. +- You must have suitable [permissions](../permissions.md). -On this page you can: +You can delete packages by using [the API](../../api/packages.md#delete-a-project-package) or the UI. -- See the extended package information, including metadata. This is unique to -each package type and will display different information for different types. -- View quick installation and registry setup instructions. These are a shortcut -for users who have already set up the Package Registry and just want quick -installation instructions. -- View the package activity, including when and how a package was published. -- View and download the contents of the package. Outside of installing a -package via a manager, you can also download the files individually. -- Delete the package (if you have suitable [permissions](../permissions.md)). +To delete a package in the UI: -### Build packages via GitLab CI/CD +1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Find the name of the package you want to delete. +1. Click **Delete**. -Some of the supported packages can be built via [GitLab CI/CD](./../../ci/README.md) -using the `CI_JOB_TOKEN`. If a package is built this way, then extended activity -information is displayed on the package details page: +The package is permanently deleted. -![Package CI/CD activity](img/package_activity_v12_10.png) +## Disable the Package Registry -You can view which pipeline published the package, as well as the commit and -user who triggered it. To see if a package type supports being built via CI/CD, -check the specific documentation for your package type. +The Package Registry is automatically enabled. + +If you are using a self-managed instance of GitLab, your administrator can remove +the menu item, **{package}** **Packages & Registries**, from the GitLab sidebar. For more information, +see the [administration documentation](../../administration/packages/index.md). + +You can also remove the Package Registry for your project specifically: + +1. In your project, go to **{settings}** **Settings > General**. +1. Expand the **Visibility, project features, permissions** section and disable the + **Packages** feature. +1. Click **Save changes**. + +The **{package}** **Packages & Registries > Package Registry** entry is removed from the sidebar. + +## Package workflows + +Learn how to use the GitLab Package Registry to build your own custom package workflow. + +- [Use a project as a package registry](./workflows/project_registry.md) to publish all of your packages to one project. +- Publish multiple different packages from one [monorepo project](./workflows/monorepo.md). ## Suggested contributions @@ -125,10 +126,3 @@ are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_reques | [RubyGems](https://gitlab.com/gitlab-org/gitlab/-/issues/803) | Use GitLab to host your own gems. | | [SBT](https://gitlab.com/gitlab-org/gitlab/-/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | | [Vagrant](https://gitlab.com/gitlab-org/gitlab/-/issues/36899) | Securely host your Vagrant boxes in local repositories. | - -## Package workflows - -Learning how to use the GitLab Package Registry will help you build your own custom package workflow. - -- [Use a project as a package registry](./workflows/project_registry.md) to publish all of your packages to one project. -- [Working with a monorepo](./workflows/monorepo.md): Learn how to publish multiple different packages from one monorepo project. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 2d40a623fb8..b194b7d837a 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -23,7 +23,7 @@ After the Packages feature is enabled, the Maven Repository will be available fo all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -821,6 +821,16 @@ user's home location (in this case the user is `root` since it runs in a Docker container), and Maven will use the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). +### Version validation + +The version string is validated using the following regex. + +```ruby +\A(\.?[\w\+-]+\.?)+\z +``` + +You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/rrLQqUXjfKEoL6). + ## Troubleshooting ### Useful Maven command line options diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 4d60c227ccd..390b2c28e10 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -25,7 +25,7 @@ This option is available only if your GitLab administrator has After the NPM registry is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. @@ -238,7 +238,8 @@ The regex that is used for naming is validating all package names from all packa It allows for capital letters, while NPM does not, and allows for almost all of the characters NPM allows with a few exceptions (`~` is not allowed). -NOTE: **Note:** Capital letters are needed because the scope is required to be +NOTE: **Note:** +Capital letters are needed because the scope is required to be identical to the top level namespace of the project. So, for example, if your project path is `My-Group/project-foo`, your package must be named `@My-Group/any-package-name`. `@my-group/any-package-name` will not work. @@ -306,10 +307,12 @@ stages: deploy: stage: deploy script: - - echo '//gitlab.com/api/v4/projects//packages/npm/:_authToken=${CI_JOB_TOKEN}'>.npmrc + - echo '//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}'>.npmrc - npm publish ``` +Learn more about [using `CI_JOB_TOKEN` to authenticate to the GitLab NPM registry](#authenticating-with-a-ci-job-token). + ## Troubleshooting ### Error running yarn with NPM registry diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 6ada68dcceb..4ee5e5c4627 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -63,7 +63,7 @@ This option is available only if your GitLab administrator has After the NuGet Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 6f6609d82b1..615741cc303 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -28,7 +28,7 @@ This option is available only if your GitLab administrator has After the PyPi Repository is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 21435c41d68..ba62cf81847 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -25,7 +25,7 @@ To add or import a user, you can follow the ## Principles behind permissions -See our [product handbook on permissions](https://about.gitlab.com/handbook/product/#permissions-in-gitlab) +See our [product handbook on permissions](https://about.gitlab.com/handbook/product/gitlab-the-product/#permissions-in-gitlab). ## Instance-wide user permissions @@ -47,7 +47,7 @@ The following table depicts the various user permission levels in a project. |---------------------------------------------------|---------|------------|-------------|----------|--------| | Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | View Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | @@ -86,7 +86,7 @@ The following table depicts the various user permission levels in a project. | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create/edit requirements **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Pull [packages](packages/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Publish [packages](packages/index.md) **(PREMIUM)**| | | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) **(PREMIUM)**| | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | | Create/edit/delete [Releases](project/releases/index.md)| | | ✓ | ✓ | ✓ | | Create new branches | | | ✓ | ✓ | ✓ | @@ -120,6 +120,7 @@ The following table depicts the various user permission levels in a project. | Rewrite/remove Git tags | | | ✓ | ✓ | ✓ | | Manage Feature Flags **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | +| Run CI/CD pipeline against a protected branch | | | ✓ (*5*) | ✓ | ✓ | | Use environment terminals | | | | ✓ | ✓ | | Run Web IDE's Interactive Web Terminals **(ULTIMATE ONLY)** | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | @@ -139,7 +140,10 @@ The following table depicts the various user permission levels in a project. | Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | | Remove GitLab Pages | | | | ✓ | ✓ | | Manage clusters | | | | ✓ | ✓ | +| Manage Project Operations | | | | ✓ | ✓ | | View Pods logs | | | | ✓ | ✓ | +| Read Terraform state | | | ✓ | ✓ | ✓ | +| Manage Terraform state | | | | ✓ | ✓ | | Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | | Manage Error Tracking | | | | ✓ | ✓ | @@ -154,6 +158,7 @@ The following table depicts the various user permission levels in a project. | Remove project | | | | | ✓ | | Archive project | | | | | ✓ | | Delete issues | | | | | ✓ | +| Delete pipelines | | | | | ✓ | | Delete merge request | | | | | ✓ | | Disable notification emails | | | | | ✓ | | Force push to protected branches (*4*) | | | | | | @@ -246,6 +251,7 @@ group. | Create project in group | | | ✓ (3) | ✓ (3) | ✓ (3) | | Share (invite) groups with groups | | | | | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| Create/edit/delete iterations | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | @@ -267,8 +273,8 @@ group. | View Issues analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Billing **(FREE ONLY)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | -| View Usage Quotas **(FREE ONLY)** | ✓ | ✓ | ✓ | ✓ | ✓ (4) | +| View Billing **(FREE ONLY)** | | | | | ✓ (4) | +| View Usage Quotas **(FREE ONLY)** | | | | | ✓ (4) | 1. Groups can be set to [allow either Owners or Owners and Maintainers to create subgroups](group/subgroups/index.md#creating-a-subgroup) @@ -281,7 +287,7 @@ group. ### Subgroup permissions When you add a member to a subgroup, they inherit the membership and -permission level from the parent group. This model allows access to +permission level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. To learn more, read through the documentation on diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index 3c6f2989091..a70d11438f4 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -35,7 +35,8 @@ As an administrator, you can delete a user account by: - **Delete user and contributions** to delete the user and their associated records. -DANGER: **Danger:** Using the **Delete user and contributions** option may result +DANGER: **Danger:** +Using the **Delete user and contributions** option may result in removing more data than intended. Please see [associated records](#associated-records) below for additional details. diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index bfcaeaf6a15..4f769f9a671 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -5,9 +5,9 @@ group: Access info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Two-Factor Authentication +# Two-factor authentication -Two-factor Authentication (2FA) provides an additional level of security to your +Two-factor authentication (2FA) provides an additional level of security to your GitLab account. Once enabled, in addition to supplying your username and password to login, you'll be prompted for a code generated by your one time password authenticator. For example, a password manager on one of your devices. @@ -62,7 +62,7 @@ To enable 2FA: 1. Click **Submit**. If the pin you entered was correct, you'll see a message indicating that -Two-Factor Authentication has been enabled, and you'll be presented with a list +two-factor authentication has been enabled, and you'll be presented with a list of [recovery codes](#recovery-codes). Make sure you download them and keep them in a safe place. diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 663a2888ee7..7a871afd861 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -22,7 +22,7 @@ See the [authentication topic](../../topics/authentication/index.md) for more de ### Unknown sign-in -GitLab will notify you if a sign-in occurs that is from an unknown IP address. +GitLab will notify you if a sign-in occurs that is from an unknown IP address or device. See [Unknown Sign-In Notification](unknown_sign_in_notification.md) for more details. ## User profile diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index ee228050945..dbf486e399e 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -187,7 +187,7 @@ To minimize the number of notifications that do not require any action, from [Gi | Remove milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected | | New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher | | Failed pipeline | The author of the pipeline | -| Fixed pipeline | The author of the pipeline. Disabled by default. To activate it you must [enable the `ci_pipeline_fixed_notifications` feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-in-development). | +| Fixed pipeline | The author of the pipeline. Enabled by default. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24309) in GitLab 13.1. Administrators can disable this notification option using the `ci_pipeline_fixed_notifications` [feature flag](../../administration/feature_flags.md). | | Successful pipeline | The author of the pipeline, if they have the custom notification setting for successful pipelines set. If the pipeline failed previously, a `Fixed pipeline` message will be sent for the first successful pipeline after the failure, then a `Successful pipeline` message for any further successful pipelines. | | New epic **(ULTIMATE)** | | | Close epic **(ULTIMATE)** | | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index e2c3dc74cf1..59ca124f566 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -19,6 +19,7 @@ Personal access tokens expire on the date you define, at midnight UTC. - GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that will expire in under seven days. The owners of these tokens are notified by email. - In GitLab Ultimate, administrators may [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only). +- In GitLab Ultimate, administrators may [toggle enforcement of personal access token expiry](../admin_area/settings/account_and_limit_settings.md#optional-enforcement-of-personal-access-token-expiry-ultimate-only). For examples of how you can use a personal access token to authenticate with the API, see the following section from our [API Docs](../../api/README.md#personalproject-access-tokens). @@ -43,6 +44,10 @@ profile. At any time, you can revoke any personal access token by clicking the respective **Revoke** button under the **Active Personal Access Token** area. +### Token activity + +You can see when a token was last used from the **Personal Access Tokens** page. Updates to the token usage is fixed at once per 24 hours. Requests to [API resources](../../api/api_resources.md) and the [GraphQL API](../../api/graphql/index.md) will update a token's usage. + ## Limiting scopes of a personal access token Personal access tokens can be created with one or more scopes that allow various diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index a5fa3cf373f..b94ae958d3b 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -63,7 +63,10 @@ Dark theme currently only works with the 'Dark' syntax highlighting. NOTE: **Note:** GitLab uses the [rouge Ruby library](http://rouge.jneen.net/ "Rouge website") -for syntax highlighting. For a list of supported languages visit the rouge website. +for syntax highlighting outside of any Editor context. The WebIDE (like Snippets) +uses [Monaco Editor](https://microsoft.github.io/monaco-editor/) and it's provided [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for +syntax highlighting. For a list of supported languages, visit the documentation of +the respective libraries. Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. diff --git a/doc/user/profile/unknown_sign_in_notification.md b/doc/user/profile/unknown_sign_in_notification.md index 200358bb050..6a6820bb2d4 100644 --- a/doc/user/profile/unknown_sign_in_notification.md +++ b/doc/user/profile/unknown_sign_in_notification.md @@ -9,16 +9,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27211) in GitLab 13.0. -When a user successfully signs in from a previously unknown IP address, +NOTE: **Note:** +This feature is enabled by default for self-managed instances. Administrators may disable this feature +through the [Sign-in restrictions](../admin_area/settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins) section of the UI. +The feature is always enabled on GitLab.com. + +When a user successfully signs in from a previously unknown IP address or device, GitLab notifies the user by email. In this way, GitLab proactively alerts users of potentially malicious or unauthorized sign-ins. -There are two methods used to identify a known sign-in: +There are several methods used to identify a known sign-in. All methods must fail +for a notification email to be sent. - Last sign-in IP: The current sign-in IP address is checked against the last sign-in IP address. - Current active sessions: If the user has an existing active session from the same IP address. See [Active Sessions](active_sessions.md). +- Cookie: After successful sign in, an encrypted cookie is stored in the browser. + This cookie is set to expire 14 days after the last successful sign in. ## Example email diff --git a/doc/user/project/bulk_editing.md b/doc/user/project/bulk_editing.md index 6d091c431a2..c4a6aea807c 100644 --- a/doc/user/project/bulk_editing.md +++ b/doc/user/project/bulk_editing.md @@ -14,7 +14,7 @@ For more details, see If you want to update attributes across multiple issues or merge requests, you can do it by bulk editing them, that is, editing them together. -![Bulk editing](img/bulk-editing.png) +![Bulk editing](img/bulk-editing_v13_2.png) ## Bulk edit issues at the project level @@ -25,8 +25,12 @@ When bulk editing issues in a project, you can edit the following attributes: - Status (open/closed) - Assignee +- Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in + [GitLab Premium](https://about.gitlab.com/pricing/) 13.2.) **(PREMIUM)** - Milestone - Labels +- Health status ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218395) in + [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2.) **(ULTIMATE)** - Subscriptions To update multiple project issues at the same time: diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index d0cba729e35..65f1c59f4ca 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -13,6 +13,11 @@ GitLab offers integrated cluster creation for the following Kubernetes providers GitLab can also integrate with any standard Kubernetes provider, either on-premise or hosted. +NOTE: **Note:** +Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) +and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) +in a few clicks. + TIP: **Tip:** Every new Google Cloud Platform (GCP) account receives [$300 in credit upon sign up](https://console.cloud.google.com/freetrial), and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's @@ -23,7 +28,7 @@ Google Kubernetes Engine Integration. All you have to do is [follow this link](h Before [adding a Kubernetes cluster](#create-new-cluster) using GitLab, you need: - GitLab itself. Either: - - A GitLab.com [account](https://about.gitlab.com/pricing/#gitlab-com). + - A [GitLab.com account](https://about.gitlab.com/pricing/#gitlab-com). - A [self-managed installation](https://about.gitlab.com/pricing/#self-managed) with GitLab version 12.5 or later. This will ensure the GitLab UI can be used for cluster creation. - The following GitLab access: @@ -52,14 +57,10 @@ to manage the newly created cluster. NOTE: **Note:** Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. -When you install Helm into your cluster, the `tiller` service account -is created with `cluster-admin` privileges in the `gitlab-managed-apps` -namespace. - -This service account will be: - -- Added to the installed Helm Tiller. -- Used by Helm to install and run [GitLab managed applications](index.md#installing-applications). +The first time you install an application into your cluster, the `tiller` service +account is created with `cluster-admin` privileges in the +`gitlab-managed-apps` namespace. This service account will be used by Helm to +install and run [GitLab managed applications](index.md#installing-applications). Helm will also create additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application @@ -88,8 +89,8 @@ GitLab creates the following resources for RBAC clusters. | `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | | `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new cluster | | `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | | Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | | Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | | Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | @@ -103,8 +104,8 @@ GitLab creates the following resources for ABAC clusters. |:----------------------|:---------------------|:-------------------------------------|:---------------------------| | `gitlab` | `ServiceAccount` | `default` namespace | Creating a new cluster | | `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm charts | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm charts | | Environment namespace | `Namespace` | Contains all environment-specific resources | Deploying to a cluster | | Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | | Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | @@ -126,7 +127,7 @@ arbitrary images as they effectively have root access. If you don't want to use GitLab Runner in privileged mode, either: - Use shared Runners on GitLab.com. They don't have this security issue. -- Set up your own Runners using configuration described at +- Set up your own Runners using the configuration described at [Shared Runners](../../gitlab_com/index.md#shared-runners). This involves: 1. Making sure that you don't have it installed via [the applications](index.md#installing-applications). @@ -135,23 +136,26 @@ If you don't want to use GitLab Runner in privileged mode, either: ## Create new cluster -New clusters can be created using GitLab for: +New clusters can be created using GitLab on Google Kubernetes Engine (GKE) or +Amazon Elastic Kubernetes Service (EKS) at the project, group, or instance level: -- [Google Kubernetes Engine (GKE)](add_gke_clusters.md). -- [Amazon Elastic Kubernetes Service (EKS)](add_eks_clusters.md). +1. Navigate to your: + - Project's **{cloud-gear}** **Operations > Kubernetes** page, for a project-level cluster. + - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. + - **{admin}** **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Click **Add Kubernetes cluster**. +1. Click the **Create new cluster** tab. +1. Click either **Amazon EKS** or **Google GKE**, and follow the instructions for your desired service: + - [Amazon EKS](add_eks_clusters.md#new-eks-cluster). + - [Google GKE](add_gke_clusters.md#creating-the-cluster-on-gke). ## Add existing cluster If you have an existing Kubernetes cluster, you can add it to a project, group, or instance. -For more information, see information for adding an: - -- [Existing Kubernetes cluster](#existing-kubernetes-cluster), including GKE clusters. -- [Existing EKS cluster](add_eks_clusters.md#existing-eks-cluster). - NOTE: **Note:** Kubernetes integration is not supported for arm64 clusters. See the issue -[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64044) for details. +[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838) for details. ### Existing Kubernetes cluster @@ -214,9 +218,9 @@ To add a Kubernetes cluster to your project, group, or instance: kind: ClusterRole name: cluster-admin subjects: - - kind: ServiceAccount - name: gitlab-admin - namespace: kube-system + - kind: ServiceAccount + name: gitlab-admin + namespace: kube-system ``` 1. Apply the service account and cluster role binding to your cluster: @@ -297,14 +301,15 @@ to install some [pre-defined applications](index.md#installing-applications). When connecting a cluster via GitLab integration, you may specify whether the cluster is RBAC-enabled or not. This will affect how GitLab interacts with the -cluster for certain operations. If you **did not** check the "RBAC-enabled cluster" +cluster for certain operations. If you did *not* check the **RBAC-enabled cluster** checkbox at creation time, GitLab will assume RBAC is disabled for your cluster when interacting with it. If so, you must disable RBAC on your cluster for the integration to work properly. -![rbac](img/rbac.png) +![rbac](img/rbac_v13_1.png) -NOTE: **Note**: Disabling RBAC means that any application running in the cluster, +NOTE: **Note:** +Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](index.md#security-implications), and may not be desirable. @@ -320,17 +325,20 @@ kubectl create clusterrolebinding permissive-binding \ ## Enabling or disabling integration -After you have successfully added your cluster information, you can enable the -Kubernetes cluster integration: - -1. Click the **Enabled/Disabled** switch -1. Hit **Save** for the changes to take effect +The Kubernetes cluster integration enables after you have successfully either created +a new cluster or added an existing one. To disable Kubernetes cluster integration: -To disable the Kubernetes cluster integration, follow the same procedure. +1. Navigate to your: + - Project's **{cloud-gear}** **Operations > Kubernetes** page, for a project-level cluster. + - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. + - **{admin}** **Admin Area >** **{cloud-gear}** **Kubernetes** page, for an instance-level cluster. +1. Click on the name of the cluster. +1. Click the **GitLab Integration** toggle. +1. Click **Save changes**. ## Removing integration -To remove the Kubernetes cluster integration from your project, either: +To remove the Kubernetes cluster integration from your project, first navigate to the **Advanced Settings** tab of the cluster details page and either: - Select **Remove integration**, to remove only the Kubernetes integration. - [From GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/26815), select diff --git a/doc/user/project/clusters/img/rbac.png b/doc/user/project/clusters/img/rbac.png deleted file mode 100644 index 517e4f7ca44..00000000000 Binary files a/doc/user/project/clusters/img/rbac.png and /dev/null differ diff --git a/doc/user/project/clusters/img/rbac_v13_1.png b/doc/user/project/clusters/img/rbac_v13_1.png new file mode 100644 index 00000000000..2772af9ff89 Binary files /dev/null and b/doc/user/project/clusters/img/rbac_v13_1.png differ diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 961a9fda5ff..ddcfd376d89 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -12,15 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in > GitLab 11.11 for [instances](../../instance/clusters/index.md). -GitLab provides many features with a Kubernetes integration. Kubernetes can be -integrated with projects, but also: - -- [Groups](../../group/clusters/index.md). -- [Instances](../../instance/clusters/index.md). - -NOTE: **Scalable app deployment with GitLab and Google Cloud Platform** -[Watch the webcast](https://about.gitlab.com/webcast/scalable-app-deploy/) and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) in a few clicks. - ## Overview Using the GitLab project Kubernetes integration, you can: @@ -28,17 +19,26 @@ Using the GitLab project Kubernetes integration, you can: - Use [Review Apps](../../../ci/review_apps/index.md). - Run [pipelines](../../../ci/pipelines/index.md). - [Deploy](#deploying-to-a-kubernetes-cluster) your applications. -- Detect and [monitor Kubernetes](#kubernetes-monitoring). +- Detect and [monitor Kubernetes](#monitoring-your-kubernetes-cluster). - Use it with [Auto DevOps](#auto-devops). - Use [Web terminals](#web-terminals). - Use [Deploy Boards](#deploy-boards-premium). **(PREMIUM)** - Use [Canary Deployments](#canary-deployments-premium). **(PREMIUM)** -- View [Logs](#logs). +- View [Logs](#viewing-pod-logs). - Run serverless workloads on [Kubernetes with Knative](serverless/index.md). +Besides integration at the project level, Kubernetes clusters can also be +integrated at the [group level](../../group/clusters/index.md) or +[GitLab instance level](../../instance/clusters/index.md). + +## Setting up + ### Supported cluster versions -GitLab is committed to support at least two production-ready Kubernetes minor versions at any given time. We regularly review the versions we support, and provide a four-month deprecation period before we remove support of a specific version. The range of supported versions is based on the evaluation of: +GitLab is committed to support at least two production-ready Kubernetes minor +versions at any given time. We regularly review the versions we support, and +provide a four-month deprecation period before we remove support of a specific +version. The range of supported versions is based on the evaluation of: - Our own needs. - The versions supported by major managed Kubernetes providers. @@ -55,80 +55,83 @@ Currently, GitLab supports the following Kubernetes versions: NOTE: **Note:** Some GitLab features may support versions outside the range provided here. -### Deploy Boards **(PREMIUM)** - -GitLab's Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes, -displaying the status of the pods in the deployment. Developers and other -teammates can view the progress and status of a rollout, pod by pod, in the -workflow they already use without any need to access Kubernetes. - -[Read more about Deploy Boards](../deploy_boards.md) - -### Canary Deployments **(PREMIUM)** - -Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) -and visualize your canary deployments right inside the Deploy Board, without -the need to leave GitLab. - -[Read more about Canary Deployments](../canary_deployments.md) +### Adding and removing clusters -### Logs - -GitLab makes it easy to view the logs of running pods in connected Kubernetes clusters. By displaying the logs directly in GitLab, developers can avoid having to manage console tools or jump to a different interface. - -[Read more about Kubernetes logs](kubernetes_pod_logs.md) +See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how +to: -### Kubernetes monitoring +- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service + (EKS) using GitLab's UI. +- Add an integration to an existing cluster from any Kubernetes platform. -Automatically detect and monitor Kubernetes metrics. Automatic monitoring of -[NGINX Ingress](../integrations/prometheus_library/nginx.md) is also supported. +### Multiple Kubernetes clusters -[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) +> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3 +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab core in 13.2. -### Auto DevOps +You can associate more than one Kubernetes cluster to your +project. That way you can have different clusters for different environments, +like dev, staging, production, and so on. -Auto DevOps automatically detects, builds, tests, deploys, and monitors your -applications. +Simply add another cluster, like you did the first time, and make sure to +[set an environment scope](#setting-the-environment-scope-premium) that will +differentiate the new cluster with the rest. -To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) -you will need the Kubernetes project integration enabled. +#### Setting the environment scope **(PREMIUM)** -[Read more about Auto DevOps](../../../topics/autodevops/index.md) +When adding more than one Kubernetes cluster to your project, you need to differentiate +them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the +[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. -NOTE: **Note** -Kubernetes clusters can be used without Auto DevOps. +The default environment scope is `*`, which means all jobs, regardless of their +environment, will use that cluster. Each scope can only be used by a single +cluster in a project, and a validation error will occur if otherwise. +Also, jobs that don't have an environment keyword set will not be able to access any cluster. -### Web terminals +For example, let's say the following Kubernetes clusters exist in a project: -> Introduced in GitLab 8.15. +| Cluster | Environment scope | +| ----------- | ----------------- | +| Development | `*` | +| Production | `production` | -When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) -support to your [environments](../../../ci/environments/index.md). This is based on the `exec` functionality found in -Docker and Kubernetes, so you get a new shell session within your existing -containers. To use this integration, you should deploy to Kubernetes using -the deployment variables above, ensuring any deployments, replica sets, and -pods are annotated with: +And the following environments are set in +[`.gitlab-ci.yml`](../../../ci/yaml/README.md): -- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` -- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` +```yaml +stages: + - test + - deploy -`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of -the CI variables. +test: + stage: test + script: sh test -You must be the project owner or have `maintainer` permissions to use terminals. Support is limited -to the first container in the first pod of your environment. +deploy to staging: + stage: deploy + script: make deploy + environment: + name: staging + url: https://staging.example.com/ -## Adding and removing clusters +deploy to production: + stage: deploy + script: make deploy + environment: + name: production + url: https://example.com/ +``` -See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how -to: +The result will then be: -- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service - (EKS) using GitLab's UI. -- Add an integration to an existing cluster from any Kubernetes platform. +- The Development cluster details will be available in the `deploy to staging` + job. +- The production cluster details will be available in the `deploy to production` + job. +- No cluster details will be available in the `test` job because it doesn't + define any environment. -## Cluster configuration +## Configuring your Kubernetes cluster After [adding a Kubernetes cluster](add_remove_clusters.md) to GitLab, read this section that covers important considerations for configuring Kubernetes clusters with GitLab. @@ -203,72 +206,6 @@ you can either: - Create an `A` record that points to the Ingress IP address with your domain provider. - Enter a wildcard DNS address using a service such as nip.io or xip.io. For example, `192.168.1.1.xip.io`. -### Setting the environment scope **(PREMIUM)** - -When adding more than one Kubernetes cluster to your project, you need to differentiate -them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the -[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. - -The default environment scope is `*`, which means all jobs, regardless of their -environment, will use that cluster. Each scope can only be used by a single -cluster in a project, and a validation error will occur if otherwise. -Also, jobs that don't have an environment keyword set will not be able to access any cluster. - -For example, let's say the following Kubernetes clusters exist in a project: - -| Cluster | Environment scope | -| ----------- | ----------------- | -| Development | `*` | -| Production | `production` | - -And the following environments are set in -[`.gitlab-ci.yml`](../../../ci/yaml/README.md): - -```yaml -stages: -- test -- deploy - -test: - stage: test - script: sh test - -deploy to staging: - stage: deploy - script: make deploy - environment: - name: staging - url: https://staging.example.com/ - -deploy to production: - stage: deploy - script: make deploy - environment: - name: production - url: https://example.com/ -``` - -The result will then be: - -- The Development cluster details will be available in the `deploy to staging` - job. -- The production cluster details will be available in the `deploy to production` - job. -- No cluster details will be available in the `test` job because it doesn't - define any environment. - -### Multiple Kubernetes clusters **(PREMIUM)** - -> Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. - -With GitLab Premium, you can associate more than one Kubernetes cluster to your -project. That way you can have different clusters for different environments, -like dev, staging, production, and so on. - -Simply add another cluster, like you did the first time, and make sure to -[set an environment scope](#setting-the-environment-scope-premium) that will -differentiate the new cluster with the rest. - ## Installing applications GitLab can install and manage some applications like Helm, GitLab Runner, Ingress, @@ -277,6 +214,19 @@ installing, upgrading, uninstalling, and troubleshooting applications for your project cluster, see [GitLab Managed Apps](../../clusters/applications.md). +## Auto DevOps + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. + +To make full use of Auto DevOps (Auto Deploy, Auto Review Apps, and +Auto Monitoring) you will need the Kubernetes project integration enabled. + +[Read more about Auto DevOps](../../../topics/autodevops/index.md) + +NOTE: **Note:** +Kubernetes clusters can be used without Auto DevOps. + ## Deploying to a Kubernetes cluster A Kubernetes cluster can be the destination for a deployment job. If @@ -325,10 +275,59 @@ For **non**-GitLab-managed clusters, the namespace can be customized using [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) in `.gitlab-ci.yml`. -NOTE: **Note:** When using a [GitLab-managed cluster](#gitlab-managed-clusters), the +NOTE: **Note:** +When using a [GitLab-managed cluster](#gitlab-managed-clusters), the namespaces are created automatically prior to deployment and [can not be customized](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +### Integrations + +#### Canary Deployments **(PREMIUM)** + +Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) +and visualize your canary deployments right inside the Deploy Board, without +the need to leave GitLab. + +[Read more about Canary Deployments](../canary_deployments.md) + +#### Deploy Boards **(PREMIUM)** + +GitLab's Deploy Boards offer a consolidated view of the current health and +status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes, +displaying the status of the pods in the deployment. Developers and other +teammates can view the progress and status of a rollout, pod by pod, in the +workflow they already use without any need to access Kubernetes. + +[Read more about Deploy Boards](../deploy_boards.md) + +#### Viewing pod logs + +GitLab makes it easy to view the logs of running pods in connected Kubernetes +clusters. By displaying the logs directly in GitLab, developers can avoid having +to manage console tools or jump to a different interface. + +[Read more about Kubernetes logs](kubernetes_pod_logs.md) + +#### Web terminals + +> Introduced in GitLab 8.15. + +When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) +support to your [environments](../../../ci/environments/index.md). This is based +on the `exec` functionality found in Docker and Kubernetes, so you get a new +shell session within your existing containers. To use this integration, you +should deploy to Kubernetes using the deployment variables above, ensuring any +deployments, replica sets, and pods are annotated with: + +- `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` +- `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` + +`$CI_ENVIRONMENT_SLUG` and `$CI_PROJECT_PATH_SLUG` are the values of +the CI variables. + +You must be the project owner or have `maintainer` permissions to use terminals. +Support is limited to the first container in the first pod of your environment. + ### Troubleshooting Before the deployment jobs starts, GitLab creates the following specifically for @@ -353,15 +352,23 @@ Reasons for failure include: [`environment:name`](../../../ci/environments/index.md#defining-environments). If your job has no `environment:name` set, it will not be passed the Kubernetes credentials. -NOTE: **NOTE:** +NOTE: **Note:** Project-level clusters upgraded from GitLab 12.0 or older may be configured in a way that causes this error. Ensure you deselect the [GitLab-managed cluster](#gitlab-managed-clusters) option if you want to manage namespaces and service accounts yourself. -## Monitoring your Kubernetes cluster **(ULTIMATE)** +## Monitoring your Kubernetes cluster + +Automatically detect and monitor Kubernetes metrics. Automatic monitoring of +[NGINX Ingress](../integrations/prometheus_library/nginx.md) is also supported. + +[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) + +### Visualizing cluster health -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/208224) to GitLab core in 13.2. When [Prometheus is deployed](#installing-applications), GitLab will automatically monitor the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start. diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 509be4ed0a8..ee642dc18cf 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -13,9 +13,9 @@ GitLab makes it easy to view the logs of running pods in [connected Kubernetes c By displaying the logs directly in GitLab in the **Log Explorer**, developers can avoid managing console tools or jumping to a different interface. -NOTE: **Kubernetes + GitLab** +NOTE: **Note:** +[Learn more about Kubernetes + GitLab](https://about.gitlab.com/solutions/kubernetes/). Everything you need to build, test, deploy, and run your application at scale. -[Learn more](https://about.gitlab.com/solutions/kubernetes/). ## Overview diff --git a/doc/user/project/clusters/runbooks/img/helm-install.png b/doc/user/project/clusters/runbooks/img/helm-install.png deleted file mode 100644 index e67cf317ec1..00000000000 Binary files a/doc/user/project/clusters/runbooks/img/helm-install.png and /dev/null differ diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 92ef35ad93f..a592d59f964 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -42,9 +42,6 @@ To create an executable runbook, you will need: - **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications. The simplest way to get started is to add a cluster using one of [GitLab's integrations](../add_remove_clusters.md#create-new-cluster). -- **Helm Tiller** - Helm is a package manager for Kubernetes and is required to - install all the other applications. It's installed in its own pod inside the - cluster which can run the Helm CLI in a safe environment. - **Ingress** - Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications. - **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user @@ -68,13 +65,8 @@ the components outlined above and the pre-loaded demo runbook. 1. Add a Kubernetes cluster to your project by following the steps outlined in [Create new cluster](../add_remove_clusters.md#create-new-cluster). -1. After the cluster has been provisioned in GKE, click the **Install** button - next to the **Helm Tiller** application to install Helm Tiller. - ![install helm](img/helm-install.png) - -1. After Helm Tiller has been installed successfully, click the **Install** button next - to the **Ingress** application. +1. Click the **Install** button next to the **Ingress** application to install Ingress. ![install ingress](img/ingress-install.png) diff --git a/doc/user/project/clusters/securing.md b/doc/user/project/clusters/securing.md new file mode 100644 index 00000000000..b4c20cb8dbc --- /dev/null +++ b/doc/user/project/clusters/securing.md @@ -0,0 +1,154 @@ +--- +stage: Defend +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Securing your deployed applications + +GitLab makes it easy to secure applications deployed in [connected Kubernetes clusters](index.md). +You can benefit from the protection of a [Web Application Firewall](../../../topics/web_application_firewall/quick_start_guide.md), +[Network Policies](../../../topics/autodevops/stages.md#network-policy), +or even [Container Host Security](../../clusters/applications.md#install-falco-using-gitlab-cicd). + +This page contains full end-to-end steps and instructions to connect your cluster to GitLab and +install these features, whether or not your applications are deployed through GitLab CI/CD. If you +use [Auto DevOps](../../../topics/autodevops/index.md) +to build and deploy your application with GitLab, see the documentation for the respective +[GitLab Managed Applications](../../clusters/applications.md) +above. + +## Overview + +At a high level, the required steps include the following: + +- Connect the cluster to GitLab. +- Set up one or more runners. +- Set up a cluster management project. +- Install a Web Application Firewall, Network Policies, and/or Container Host + Security. +- Install Prometheus to get statistics and metrics in the + [threat monitoring](../../application_security/threat_monitoring/) + dashboard. + +### Requirements + +Minimum requirements (depending on the GitLab Manage Application you want to install): + +- Your cluster is connected to GitLab (ModSecurity, Cilium, and Falco). +- At least one GitLab Runner is installed (Cilium and Falco only). + +### Understanding how GitLab Managed Apps are installed + +You install GitLab Managed Apps from the GitLab web interface with a one-click setup process. GitLab +uses Sidekiq (a background processing service) to facilitate this. + +```mermaid + sequenceDiagram + autonumber + GitLab->>+Sidekiq: Install a GitLab Managed App + Sidekiq->>+Kubernetes: Helm install + Kubernetes-->>-Sidekiq: Installation complete + Sidekiq-->>-GitLab: Refresh UI +``` + +NOTE: **Note:** +This diagram uses the term _Kubernetes_ for simplicity. In practice, Sidekiq connects to a Helm +Tiller daemon running in a pod in the cluster. + +Although this installation method is easier because it's a point-and-click action in the user +interface, it's inflexible and hard to debug. When something goes wrong, you can't see the +deployment logs. The Web Application Firewall feature uses this installation method. + +However, the next generation of GitLab Managed Apps V2 ([CI/CD-based GitLab Managed Apps](https://gitlab.com/groups/gitlab-org/-/epics/2103)) +don't use Sidekiq to deploy. All the applications are deployed using a GitLab CI/CD pipeline and +therefore GitLab Runners. + +```mermaid +sequenceDiagram + autonumber + GitLab->>+GitLab: Trigger pipeline + GitLab->>+Runner: Run deployment job + Runner->>+Kubernetes: Helm install + Kubernetes-->>-Runner: Installation is complete + Runner-->>-GitLab: Report job status and update pipeline +``` + +Debugging is easier because you have access to the raw logs of these jobs (the Helm Tiller output is +available as an artifact in case of failure) and the flexibility is much better. Since these +deployments are only triggered when a pipeline is running (most likely when there's a new commit in +the cluster management repository), every action has a paper trail and follows the classic merge +request workflow (approvals, merge, deploy). The Network Policy (Cilium) Managed App and Container +Host Security (Falco) are deployed with this model. + +## Connect the cluster to GitLab + +To deploy GitLab Managed Apps to your cluster, you must first +[add your cluster](add_remove_clusters.md) +to GitLab. Then [install](../../clusters/applications.md#installing-applications) +the Web Application Firewall from the project or group Kubernetes page. + +Note that your project doesn't have to be hosted or deployed through GitLab. You can manage a +cluster independent of the applications that use the cluster. + +## Set up a GitLab Runner + +To install CI/CD-based GitLab Managed Apps, a pipeline using a GitLab Runner must be running in +GitLab. You can [install a GitLab Runner](../../clusters/applications.md#gitlab-runner) +in the Kubernetes cluster added in the previous step, or use one of the shared runners provided by +GitLab if you're using GitLab.com. + +With your cluster connected to GitLab and a GitLab Runner in place, you can proceed to the next +steps and start installing the Cilium and Falco GitLab Managed Apps to secure your applications +hosted on this cluster. + +## Create a Cluster Management Project + +A [Cluster Management Project](../../clusters/management_project.md) +is a GitLab project that contains a `.gitlab-ci.yml` file to deploy GitLab Managed Apps to your +cluster. This project runs the required charts with the Kubernetes +[`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) +privileges. + +The creation of this project starts like any other GitLab project. Use an empty +project and add a `gitlab-ci.yml` file at the root, containing this template: + +```yaml +include: + - template: Managed-Cluster-Applications.gitlab-ci.yml +``` + +To make this project a Cluster Management Project, follow these +[instructions](../../clusters/management_project.md#selecting-a-cluster-management-project). +This project can be designated as such even if your application isn't hosted on GitLab. In this +case, create a new empty project where you can select your newly created Cluster Management Project. + +## Install GitLab Container Network Policy + +GitLab Container Network Policy is based on [Cilium](https://cilium.io/). To +install the Cilium GitLab Managed App, add a +`.gitlab/managed-apps/config.yaml` file to your Cluster Management project: + +```yaml +# possible values are gke, eks or you can leave it blank +clusterType: gke + +cilium: + installed: true +``` + +Your application doesn't have to be managed or deployed by GitLab to leverage this feature. +[Read more](../../clusters/applications.md#install-cilium-using-gitlab-cicd) +about configuring Container Network Policy. + +## Install GitLab Container Host Security + +Similarly, you can install Container Host Security, based on +[Falco](https://falco.org/), in your `.gitlab/managed-apps/config.yaml`: + +```yaml +falco: + installed: true +``` + +[Read more] about configuring Container Host Security. diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 15f7e14fda9..595d8fb3895 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -392,29 +392,19 @@ want to store your package: image: python:latest stages: - - deploy production: - stage: deploy - before_script: - - pip3 install awscli --upgrade - - pip3 install aws-sam-cli --upgrade - script: - - sam build - - sam package --output-template-file packaged.yaml --s3-bucket - - sam deploy --template-file packaged.yaml --stack-name gitlabpoc --s3-bucket --capabilities CAPABILITY_IAM --region us-east-1 - environment: production - ``` +``` Let’s examine the configuration file more closely: diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 45fb313d177..6af08b06294 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -51,8 +51,6 @@ To run Knative on GitLab, you will need: 1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. The simplest way to get started is to add a cluster using GitLab's [GKE integration](../add_remove_clusters.md). The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. -1. **Helm Tiller:** Helm is a package manager for Kubernetes and is required to install - Knative. 1. **GitLab Runner:** A runner is required to run the CI jobs that will deploy serverless applications or functions onto your cluster. You can install the GitLab Runner onto the existing Kubernetes cluster. See [Installing Applications](../index.md#installing-applications) for more information. @@ -80,8 +78,8 @@ To run Knative on GitLab, you will need: NOTE: **Note:** The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. **RBAC must be enabled.** -1. [Add a Kubernetes cluster](../add_remove_clusters.md) and [install Helm](../index.md#installing-applications). -1. Once Helm has been successfully installed, scroll down to the Knative app section. Enter the domain to be used with +1. [Add a Kubernetes cluster](../add_remove_clusters.md). +1. Select the **Applications** tab and scroll down to the Knative app section. Enter the domain to be used with your application/functions (e.g. `example.com`) and click **Install**. ![install-knative](img/install-knative.png) @@ -143,24 +141,24 @@ You must do the following: labels: rbac.authorization.k8s.io/aggregate-to-edit: "true" rules: - - apiGroups: - - serving.knative.dev - resources: - - configurations - - configurationgenerations - - routes - - revisions - - revisionuids - - autoscalers - - services - verbs: - - get - - list - - create - - update - - delete - - patch - - watch + - apiGroups: + - serving.knative.dev + resources: + - configurations + - configurationgenerations + - routes + - revisions + - revisionuids + - autoscalers + - services + verbs: + - get + - list + - create + - update + - delete + - patch + - watch ``` Then run the following command: @@ -570,7 +568,7 @@ The simplest way to accomplish this is to use [Certbot to manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS. NOTE: **Note:** -The instructions below relate to installing and running Certbot on a Linux server and may not work on other operating systems. +The instructions below relate to installing and running Certbot on a Linux server that has Python 3 installed and may not work on other operating systems or with other versions of Python. 1. Install Certbot by running the [`certbot-auto` wrapper script](https://certbot.eff.org/docs/install.html#certbot-auto). @@ -580,7 +578,7 @@ The instructions below relate to installing and running Certbot on a Linux serve wget https://dl.eff.org/certbot-auto sudo mv certbot-auto /usr/local/bin/certbot-auto sudo chown root /usr/local/bin/certbot-auto - chmod 0755 /usr/local/bin/certbot-auto + sudo chmod 0755 /usr/local/bin/certbot-auto /usr/local/bin/certbot-auto --help ``` @@ -609,7 +607,7 @@ The instructions below relate to installing and running Certbot on a Linux serve using DNS challenge during authorization: ```shell - ./certbot-auto certonly --manual --preferred-challenges dns -d '*..example.com' + /usr/local/bin/certbot-auto certonly --manual --preferred-challenges dns -d '*..example.com' ``` Where `` is the namespace created by GitLab for your serverless project (composed of `--`) and @@ -835,7 +833,7 @@ The instructions below relate to installing and running Certbot on a Linux serve ## Using an older version of `gitlabktl` There may be situations where you want to run an older version of `gitlabktl`. This -requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml file.` +requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml` file. To set an older version, add `image:` to the `functions:deploy` block. For example: diff --git a/doc/user/project/code_intelligence.md b/doc/user/project/code_intelligence.md new file mode 100644 index 00000000000..e2c2cae3158 --- /dev/null +++ b/doc/user/project/code_intelligence.md @@ -0,0 +1,54 @@ +--- +type: reference +--- + +# Code Intelligence + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/1576) in GitLab 13.1. + +Code Intelligence adds code navigation features common to interactive +development environments (IDE), including: + +- Type signatures and symbol documentation. +- Go-to definition + +Code Intelligence is built into GitLab and powered by [LSIF](https://lsif.dev/) +(Language Server Index Format), a file format for precomputed code +intelligence data. + +## Configuration + +Enable code intelligence for a project by adding a GitLab CI/CD job to the project's +`.gitlab-ci.yml` which will generate the LSIF artifact: + +```yaml +code_navigation: + image: golang:1.14.0 + allow_failure: true # recommended + script: + - go get github.com/sourcegraph/lsif-go/cmd/lsif-go + - lsif-go + artifacts: + reports: + lsif: dump.lsif +``` + +The generated LSIF file must be less than 170MiB. + +After the job succeeds, code intelligence data can be viewed while browsing the code: + +![Code intelligence](img/code_intelligence_v13_1.png) + +## Language support + +Generating an LSIF file requires a language server indexer implementation for the +relevant language. + +| Language | Implementation | +|---|---| +| Go | [sourcegraph/lsif-go](https://github.com/sourcegraph/lsif-go) | +| JavaScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | +| TypeScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | + +View a complete list of [available LSIF indexers](https://lsif.dev/#implementations-server) on their website and +refer to their documentation to see how to generate an LSIF file for your specific language. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 6b81aea4b87..b35d04dfdfc 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -22,7 +22,7 @@ who is responsible for each file or path. ## Why is this useful? -Code Owners allows for a version controlled single source of +Code Owners allows for a version controlled, single source of truth file outlining the exact GitLab users or groups that own certain files or paths in a repository. Code Owners can be utilized in the merge request approval process which can streamline @@ -38,18 +38,18 @@ approval. You can use a `CODEOWNERS` file to specify users or [shared groups](members/share_project_with_groups.md) -that are responsible for certain files in a repository. +that are responsible for specific files and directories in a repository. -You can choose and add the `CODEOWNERS` file in three places: +You can choose to add the `CODEOWNERS` file in three places: - To the root directory of the repository - Inside the `.gitlab/` directory - Inside the `docs/` directory -The `CODEOWNERS` file is scoped to a branch, which means that with the -introduction of new files, the person adding the new content can +The `CODEOWNERS` file is scoped to a branch, which means that as +new files are introduced, the user adding the new content can specify themselves as a code owner, all before the new changes -get merged to the default branch. +get merged to the target branch. When a file matches multiple entries in the `CODEOWNERS` file, the users from last pattern matching the file are displayed on the @@ -67,13 +67,13 @@ The user that would show for `README.md` would be `@user2`. ## Approvals by Code Owners -Once you've set Code Owners to a project, you can configure it to +Once you've added Code Owners to a project, you can configure it to be used for merge request approvals: - As [merge request eligible approvers](merge_requests/merge_request_approvals.md#code-owners-as-eligible-approvers). - As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium). **(PREMIUM)** -NOTE: **Note**: +NOTE: **Note:** Developer or higher [permissions](../permissions.md) are required in order to approve a merge request. @@ -81,7 +81,7 @@ Once set, Code Owners are displayed in merge requests widgets: ![MR widget - Code Owners](img/code_owners_mr_widget_v12_4.png) -While the `CODEOWNERS` file can be used in addition to Merge Request [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules) +While the `CODEOWNERS` file can be used in addition to Merge Request [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules), it can also be used as the sole driver of merge request approvals (without using [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules)). To do so, create the file in one of the three locations specified above and @@ -89,28 +89,38 @@ set the code owners as required approvers for [protected branches](protected_bra Use [the syntax of Code Owners files](code_owners.md#the-syntax-of-code-owners-files) to specify the actual owners and granular permissions. -Using Code Owners in conjunction with [Protected Branches Approvals](protected_branches.md#protected-branches-approval-by-code-owners-premium) -will prevent any user who is not specified in the `CODEOWNERS` file from pushing changes -for the specified files/paths, even if their role is included in the **Allowed to push** column. -This allows for a more inclusive push strategy, as administrators don't have to restrict developers -from pushing directly to the protected branch, but can restrict pushing to certain -files where a review by Code Owners is required. +Using Code Owners in conjunction with [Protected Branches](protected_branches.md#protected-branches-approval-by-code-owners-premium) +will prevent any user who is not specified in the `CODEOWNERS` file from pushing +changes for the specified files/paths, even if their role is included in the +**Allowed to push** column. This allows for a more inclusive push strategy, as +administrators don't have to restrict developers from pushing directly to the +protected branch, but can restrict pushing to certain files where a review by +Code Owners is required. ## The syntax of Code Owners files Files can be specified using the same kind of patterns you would use -in the `.gitignore` file followed by the `@username` or email of one -or more users or by the `@name` of one or more groups that should -be owners of the file. Groups must be added as [members of the project](members/index.md), +in the `.gitignore` file followed by one or more of: + +- A user's `@username`. +- A user's email address. +- The `@name` of one or more groups that should be owners of the file. + +Groups must be added as [members of the project](members/index.md), or they will be ignored. -Starting in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/32432), you can now specify -groups or subgroups from the project's group hierarchy as potential code owners. +Starting in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/32432), +you can additionally specify groups or subgroups from the project's upper group +hierarchy as potential code owners, without having to invite them specifically +to the project. Groups outside the project's hierarchy or children beneath the +hierarchy must still be explicitly invited to the project in order to show as +Code Owners. -For example, consider the following hierarchy for a given project: +For example, consider the following hierarchy for the example project +`example_project`: ```plaintext -group >> sub-group >> sub-subgroup >> myproject >> file.md +group >> sub-group >> sub-subgroup >> example_project >> file.md ``` Any of the following groups would be eligible to be specified as code owners: @@ -119,7 +129,8 @@ Any of the following groups would be eligible to be specified as code owners: - `@group/sub-group` - `@group/sub-group/sub-subgroup` -In addition, any groups that have been invited to the project using the **Members** tool will also be recognized as eligible code owners. +In addition, any groups that have been invited to the project using the +**Members** tool will also be recognized as eligible code owners. The order in which the paths are defined is significant: the last pattern that matches a given path will be used to find the code @@ -129,7 +140,98 @@ Starting a line with a `#` indicates a comment. This needs to be escaped using `\#` to address files for which the name starts with a `#`. -Example `CODEOWNERS` file: +### Code Owners Sections **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. +> - It's deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It can be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-code-owner-sections-core-only). **(CORE ONLY)** + +Code Owner rules can be grouped into named sections. This allows for better +organization of broader categories of Code Owner rules to be applied. +Additionally, the usual guidance that only the last pattern matching the file is +applied is expanded such that the last pattern matching _for each section_ is +applied. + +For example, in a large organization, independent teams may have a common interest +in parts of the application, for instance, a payment processing company may have +"development", "security", and "compliance" teams looking after common parts of +the codebase. All three teams may need to approve changes. Although approval rules +make this possible, they apply to every merge request. Also, while Code Owners are +applied based on which files are changed, only one CODEOWNERS pattern can match per +file path. + +Using `CODEOWNERS` sections allows multiple teams that only need to approve certain +changes, to set their own independent patterns by specifying discrete sections in the +`CODEOWNERS` file. The section rules may be used for shared paths so that multiple +teams can be added as reviewers. + +Sections can be added to `CODEOWNERS` files as a new line with the name of the +section inside square brackets. Every entry following it is assigned to that +section. The following example would create 2 Code Owner rules for the "README +Owners" section: + +```plaintext +[README Owners] +README.md @user1 @user 2 +internal/README.md @user2 +``` + +Multiple sections can be used, even with matching file or directory patterns. +Reusing the same section name will group the results together under the same +section, with the most specific rule or last matching pattern being used. For +example, consider the following entries in a `CODEOWNERS` file: + +```plaintext +[Documentation] +ee/docs @gl-docs +docs @gl-docs + +[Database] +README.md @gl-database +model/db @gl-database + +[DOCUMENTATION] +README.md @gl-docs +``` + +This will result in 3 entries under the "Documentation" section header, and 2 +entries under "Database". Case is not considered when combining sections, so in +this example, entries defined under the sections "Documentation" and +"DOCUMENTATION" would be combined into one, using the case of the first instance +of the section encountered in the file. + +When assigned to a section, each code owner rule displayed in merge requests +widget is sorted under a "section" label. In the screenshot below, we can see +the rules for "Groups" and "Documentation" sections: + +![MR widget - Sectional Code Owners](img/sectional_code_owners_v13.2.png) + +#### Enable or disable Code Owner Sections **(CORE ONLY)** + +Sections is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it for your instance. + +To disable it: + +```ruby +Feature.disable(:sectional_codeowners) +``` + +To enable it: + +```ruby +Feature.enable(:sectional_codeowners) +``` + +CAUTION: **Caution:** +Disabling Sections will **not** refresh Code Owner Approval Rules on existing merge requests. + +## Example `CODEOWNERS` file ```plaintext # This is an example of a code owners file @@ -185,4 +287,17 @@ lib/ @lib-owner # If the path contains spaces, these need to be escaped like this: path\ with\ spaces/ @space-owner + +# Code Owners section: +[Documentation] +ee/docs @gl-docs +docs @gl-docs + +[Database] +README.md @gl-database +model/db @gl-database + +# This section will be joined with the [Documentation] section previously defined: +[DOCUMENTATION] +README.md @gl-docs ``` diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 10f0281d0eb..b7daca567f4 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -11,7 +11,7 @@ type: howto > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199370) from **Settings > Repository** in GitLab 12.9. > - [Added `write_registry` scope](https://gitlab.com/gitlab-org/gitlab/-/issues/22743) in GitLab 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29280) from **Settings > CI / CD** in GitLab 12.10.1. -> - [Added package registry scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) from **Settings > CI / CD** in GitLab 13.0. +> - [Added package registry scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. Deploy tokens allow you to download (`git clone`) or push and pull packages and container registry images of a project without having a user and a password. @@ -46,15 +46,17 @@ respective **Revoke** button under the 'Active deploy tokens' area. ## Limiting scopes of a deploy token -Deploy tokens can be created with two different scopes that allow various +Deploy tokens can be created with different scopes that allow various actions that a given token can perform. The available scopes are depicted in -the following table. +the following table along with GitLab version it was introduced in. -| Scope | Description | -| ----- | ----------- | -| `read_repository` | Allows read-access to the repository through `git clone` | -| `read_registry` | Allows read-access to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | -| `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | +| Scope | Description | Introduced in GitLab Version | +| ----- | ----------- | ------ | +| `read_repository` | Allows read-access to the repository through `git clone` | 10.7 | +| `read_registry` | Allows read-access to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | 10.7 | +| `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | 12.10 | +| `read_package_registry` | Allows read access to the package registry. | 13.0 | +| `write_package_registry` | Allows write access to the package registry. | 13.0 | ## Deploy token custom username @@ -96,6 +98,8 @@ pull images from your Container Registry. ### Push Container Registry images +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22743) in GitLab 12.10. + To push the container registry images, you'll need to: 1. Create a Deploy Token with `write_registry` as a scope. @@ -111,6 +115,8 @@ push images to your Container Registry. ### Read or pull packages +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. + To pull packages in the GitLab package registry, you'll need to: 1. Create a Deploy Token with `read_package_registry` as a scope. @@ -119,6 +125,8 @@ To pull packages in the GitLab package registry, you'll need to: ### Push or upload packages +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in GitLab 13.0. + To upload packages in the GitLab package registry, you'll need to: 1. Create a Deploy Token with `write_package_registry` as a scope. @@ -151,8 +159,7 @@ apply consistently when cloning the repository of related projects. There's a special case when it comes to Deploy Tokens. If a user creates one named `gitlab-deploy-token`, the username and token of the Deploy Token will be automatically exposed to the CI/CD jobs as environment variables: `CI_DEPLOY_USER` and -`CI_DEPLOY_PASSWORD`, respectively. With the GitLab Deploy Token, the -`read_registry` and `write_registry` scopes are implied. +`CI_DEPLOY_PASSWORD`, respectively. After you create the token, you can login to the Container Registry using those variables: diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 0f90c321a14..aa5987bf5f9 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -81,7 +81,7 @@ changes you made after picking the template and return it to its initial status. ![Description templates](img/description_templates.png) -## Setting a default template for merge requests and issues **(STARTER)** +## Setting a default template for merge requests and issues **(STARTER)** > - This feature was introduced before [description templates](#overview) and is available in [GitLab Starter](https://about.gitlab.com/pricing/). It can be enabled in the project's settings. > - Templates for issues were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28) in GitLab EE 8.1. diff --git a/doc/user/project/highlighting.md b/doc/user/project/highlighting.md index 2bdb0ae2706..a2740294e62 100644 --- a/doc/user/project/highlighting.md +++ b/doc/user/project/highlighting.md @@ -1,6 +1,11 @@ # Syntax Highlighting -GitLab provides syntax highlighting on all files and snippets through the [Rouge](https://rubygems.org/gems/rouge) rubygem. It will try to guess what language to use based on the file extension, which most of the time is sufficient. +GitLab provides syntax highlighting on all files through the [Rouge](https://rubygems.org/gems/rouge) Ruby gem. It will try to guess what language to use based on the file extension, which most of the time is sufficient. + +NOTE: **Note:** +The [Web IDE](web_ide/index.md) and [Snippets](../snippets.md) use [Monaco Editor](https://microsoft.github.io/monaco-editor/) +for text editing, which internally uses the [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) +library for syntax highlighting. If GitLab is guessing wrong, you can override its choice of language using the `gitlab-language` attribute in `.gitattributes`. For example, if you are working in a Prolog project and using the `.pl` file extension (which would normally be highlighted as Perl), you can add the following to your `.gitattributes` file: @@ -27,3 +32,6 @@ To disable highlighting entirely, use `gitlab-language=text`. Lots more fun shen ``` Please note that these configurations will only take effect when the `.gitattributes` file is in your default branch (usually `master`). + +NOTE: **Note:** +The Web IDE does not support `.gitattribute` files, but it's [planned for a future release](https://gitlab.com/gitlab-org/gitlab/-/issues/22014). diff --git a/doc/user/project/img/bulk-editing.png b/doc/user/project/img/bulk-editing.png deleted file mode 100644 index 8ae649e5020..00000000000 Binary files a/doc/user/project/img/bulk-editing.png and /dev/null differ diff --git a/doc/user/project/img/bulk-editing_v13_2.png b/doc/user/project/img/bulk-editing_v13_2.png new file mode 100644 index 00000000000..871cb02c01f Binary files /dev/null and b/doc/user/project/img/bulk-editing_v13_2.png differ diff --git a/doc/user/project/img/code_intelligence_v13_1.png b/doc/user/project/img/code_intelligence_v13_1.png new file mode 100644 index 00000000000..0dff27bab43 Binary files /dev/null and b/doc/user/project/img/code_intelligence_v13_1.png differ diff --git a/doc/user/project/img/sectional_code_owners_v13.2.png b/doc/user/project/img/sectional_code_owners_v13.2.png new file mode 100644 index 00000000000..894771c26af Binary files /dev/null and b/doc/user/project/img/sectional_code_owners_v13.2.png differ diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index 0d6e059f1cf..3838289aec4 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -105,7 +105,7 @@ back to both GitLab and GitHub when completed. 1. The result of the job will be visible directly from the pipeline view: - ![Security Dashboard](../../application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png) + ![Security Dashboard](../../application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png) NOTE: **Note:** If you don't commit very often to your project, you may want to use diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png index 4ab42485d0b..3c1dc44df93 100644 Binary files a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png and b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png differ diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png index 6278cb5f970..d98ad2aaa6e 100644 Binary files a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png and b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png differ diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png b/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png new file mode 100644 index 00000000000..9cbffe2bb36 Binary files /dev/null and b/doc/user/project/import/img/jira/import_issues_from_jira_form_v13_2.png differ diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png deleted file mode 100644 index bf9728e0311..00000000000 Binary files a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png and /dev/null differ diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index 0b8807bb9b3..395cca4726d 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -40,6 +40,8 @@ Make sure you have the integration set up before trying to import Jira issues. ## Import Jira issues to GitLab +> New import form [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216145) in GitLab 13.2. + To import Jira issues to a GitLab project, follow the steps below. NOTE: **Note:** @@ -47,27 +49,34 @@ Importing Jira issues is done as an asynchronous background job, which may result in delays based on import queues load, system load, or other factors. Importing large projects may take several minutes depending on the size of the import. -1. On the **{issues}** **Issues** page, click the **Import Issues** (**{import}**) button. -1. Select **Import from Jira**. - This option is only visible if you have the [correct permissions](#permissions). +1. On the **{issues}** **Issues** page, click **Import Issues** (**{import}**) **> Import from Jira**. ![Import issues from Jira button](img/jira/import_issues_from_jira_button_v12_10.png) + The **Import from Jira** option is only visible if you have the [correct permissions](#permissions). + The following form appears. + If you've previously set up the [Jira integration](../integrations/jira.md), you can now see + the Jira projects that you have access to in the dropdown. + + ![Import issues from Jira form](img/jira/import_issues_from_jira_form_v13_2.png) - ![Import issues from Jira form](img/jira/import_issues_from_jira_form_v12_10.png) +1. Click the **Import from** dropdown and select the Jira project that you wish to import issues from. - If you've previously set up the [Jira integration](../integrations/jira.md), you now see the Jira - projects that you have access to in the dropdown. + In the **Jira-GitLab user mapping template** section, the table shows to which GitLab users your Jira + users will be mapped. + If it wasn't possible to match a Jira user with a GitLab user, the dropdown defaults to the user + conducting the import. -1. Select the Jira project that you wish to import issues from. +1. To change any of the suggested mappings, click the dropdown in the **GitLab username** column and + select the user you want to map to each Jira user. - ![Import issues from Jira form](img/jira/import_issues_from_jira_projects_v12_10.png) + The dropdown may not show all the users, so use the search bar to find a specific + user in this GitLab project. + +1. Click **Continue**. You're presented with a confirmation that import has started. -1. Click **Import Issues**. You're presented with a confirmation that import has started. While the import is running in the background, you can navigate away from the import status page to the issues page, and you'll see the new issues appearing in the issues list. -1. To check the status of your import, go back to the Jira import page. - - ![Import issues from Jira button](img/jira/import_issues_from_jira_button_v12_10.png) +1. To check the status of your import, go to the Jira import page again. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 3a4e240fb6c..1e71674c16f 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -78,7 +78,7 @@ When you create a project in GitLab, you'll have access to a large number of timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more - [Kubernetes cluster integration](clusters/index.md): Connecting your GitLab project with a Kubernetes cluster - - [Feature Flags](operations/feature_flags.md): Feature flags allow you to ship a project in + - [Feature Flags](../../operations/feature_flags.md): Feature flags allow you to ship a project in different flavors by dynamically toggling certain functionality **(PREMIUM)** - [GitLab Pages](pages/index.md): Build, test, and deploy your static website with GitLab Pages @@ -104,6 +104,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Dependency List](../application_security/dependency_list/index.md): view project dependencies. **(ULTIMATE)** - [Requirements](requirements/index.md): Requirements allow you to create criteria to check your products against. **(ULTIMATE)** - [Static Site Editor](static_site_editor/index.md): quickly edit content on static websites without prior knowledge of the codebase or Git commands. +- [Code Intelligence](code_intelligence.md): code navigation features. ### Project integrations @@ -172,6 +173,24 @@ Read through the documentation on [project settings](settings/index.md). - [Export a project from GitLab](settings/import_export.md#exporting-a-project-and-its-data) - [Importing and exporting projects between GitLab instances](settings/import_export.md) +## Remove a project + +To remove a project, first navigate to the home page for that project. + +1. Navigate to **Settings > General**. +1. Expand the **Advanced** section. +1. Scroll down to the **Remove project** section. +1. Click **Remove project** +1. Confirm this action by typing in the expected text. + +### Delayed removal **(PREMIUM)** + +By default, clicking to remove a project is followed by a seven day delay. Admins can restore the project during this period of time. +This delay [may be changed by an admin](../admin_area/settings/visibility_and_access_controls.md#default-deletion-adjourned-period-premium-only). + +Admins can view all projects pending deletion. If you're an administrator, go to the top navigation bar, click **Projects > Your projects**, and then select the **Removed projects** tab. +From this tab an admin can restore any project. + ## CI/CD for external repositories **(PREMIUM)** Instead of importing a repository directly to GitLab, you can connect your repository @@ -242,14 +261,52 @@ When [renaming a user](../profile/index.md#changing-your-username), ## Use your project as a Go package -Any project can be used as a Go package including private projects in subgroups. -GitLab responds correctly to `go get` and `godoc.org` discovery requests, -including the [`go-import`](https://golang.org/cmd/go/#hdr-Remote_import_paths) -and [`go-source`](https://github.com/golang/gddo/wiki/Source-Code-Links) meta -tags, respectively. To use packages hosted in private projects with the `go get` -command, use a [`.netrc` file](https://ec.haxx.se/usingcurl-netrc.html) and a -[personal access token](../profile/personal_access_tokens.md) in the password -field. +Any project can be used as a Go package. GitLab responds correctly to `go get` +and `godoc.org` discovery requests, including the +[`go-import`](https://golang.org/cmd/go/#hdr-Remote_import_paths) and +[`go-source`](https://github.com/golang/gddo/wiki/Source-Code-Links) meta tags. + +Private projects, including projects in subgroups, can be used as a Go package, +but may require configuration to work correctly. GitLab will respond correctly +to `go get` discovery requests for projects that *are not* in subgroups, +regardless of authentication or authorization. +[Authentication](#authenticate-go-requests) is required to use a private project +in a subgroup as a Go package. Otherwise, GitLab will truncate the path for +private projects in subgroups to the first two segments, causing `go get` to +fail. + +GitLab implements its own Go proxy. This feature must be enabled by an +administrator and requires additional configuration. See [GitLab Go +Proxy](../packages/go_proxy/index.md). + +### Disable Go module features for private projects + +In Go 1.12 and later, Go queries module proxies and checksum databases in the +process of [fetching a +module](../../development/go_guide/dependencies.md#fetching). This can be +selectively disabled with `GOPRIVATE` (disable both), +[`GONOPROXY`](../../development/go_guide/dependencies.md#proxies) (disable proxy +queries), and [`GONOSUMDB`](../../development/go_guide/dependencies.md#fetching) +(disable checksum queries). + +`GOPRIVATE`, `GONOPROXY`, and `GONOSUMDB` are comma-separated lists of Go +modules and Go module prefixes. For example, +`GOPRIVATE=gitlab.example.com/my/private/project` will disable queries for that +one project, but `GOPRIVATE=gitlab.example.com` will disable queries for *all* +projects on GitLab.com. Go will not query module proxies if the module name or a +prefix of it appears in `GOPRIVATE` or `GONOPROXY`. Go will not query checksum +databases if the module name or a prefix of it appears in `GONOPRIVATE` or +`GONOSUMDB`. + +### Authenticate Go requests + +To authenticate requests to private projects made by Go, use a [`.netrc` +file](https://ec.haxx.se/usingcurl-netrc.html) and a [personal access +token](../profile/personal_access_tokens.md) in the password field. **This only +works if your GitLab instance can be accessed with HTTPS.** The `go` command +will not transmit credentials over insecure connections. This will authenticate +all HTTPS requests made directly by Go but will not authenticate requests made +through Git. For example: @@ -259,6 +316,24 @@ login password ``` +NOTE: **Note:** +On Windows, Go reads `~/_netrc` instead of `~/.netrc`. + +### Authenticate Git fetches + +If a module cannot be fetched from a proxy, Go will fall back to using Git (for +GitLab projects). Git will use `.netrc` to authenticate requests. Alternatively, +Git can be configured to embed specific credentials in the request URL, or to +use SSH instead of HTTPS (as Go always uses HTTPS to fetch Git repositories): + +```shell +# embed credentials in any request to GitLab.com: +git config --global url."https://${user}:${personal_access_token}@gitlab.example.com".insteadOf "https://gitlab.example.com" + +# use SSH instead of HTTPS: +git config --global url."git@gitlab.example.com".insteadOf "https://gitlab.example.com" +``` + ## Access project page with project ID > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53671) in GitLab 11.8. diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 3cc76beb323..4d646ee2f79 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -271,6 +271,27 @@ you defined. | `week` | 4 | | `month` | 12 | +#### `query.period_field` + +Define the timestamp field used to group "issuables". + +Supported values are: + +- `created_at` (default): Group data using the `created_at` field. +- `closed_at`: Group data using the `closed_at` field (for issues only). +- `merged_at`: Group data using the `merged_at` field (for merge requests only). + +The `period_field` is automatically set to: + +- `closed_at` if `query.issuable_state` is `closed` +- `merged_at` if `query.issuable_state` is `merged` +- `created_at` otherwise + +NOTE: **Note:** +Until [this bug](https://gitlab.com/gitlab-org/gitlab/-/issues/26911), is resolved, you may see `created_at` +in place of `merged_at`. +`created_at` will be used instead. + ### `projects` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10904) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.4. diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index de43c504b05..6d44c56743e 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -6,7 +6,6 @@ in the table below. | Field | Description | | ----- | ----------- | -| `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Bugzilla which is being linked to this GitLab project. Note that the `project_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | | `issues_url` | The URL to the issue in Bugzilla project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | | `new_issue_url` | This is the URL to create a new issue in Bugzilla for the project linked to this GitLab project. Note that the `new_issue_url` requires PRODUCT_NAME to be updated with the product/project name in Bugzilla. | diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index 4eaf3a0d4b4..7d15ae82b6f 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -11,8 +11,6 @@ To enable the Custom Issue Tracker integration in a project: | Field | Description | | --------------- | ----------- | - | **Title** | A title for the issue tracker (for example, to differentiate between instances). | - | **Description** | A name for the issue tracker (for example, to differentiate between instances). | | **Project URL** | The URL to the project in the custom issue tracker. | | **Issues URL** | The URL to the issue in the issue tracker project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. For example, `https://customissuetracker.com/project-name/:id`. | | **New issue URL** | Currently unused. Will be changed in a future release. | diff --git a/doc/user/project/integrations/generic_alerts.md b/doc/user/project/integrations/generic_alerts.md index 57c1e54e48c..3490a3f2b9e 100644 --- a/doc/user/project/integrations/generic_alerts.md +++ b/doc/user/project/integrations/generic_alerts.md @@ -18,16 +18,21 @@ create an issue with the payload in the body of the issue. You can always The entire payload will be posted in the issue discussion as a comment authored by the GitLab Alert Bot. -NOTE: **Note** -In GitLab versions 13.1 and greater, you can configure [External Prometheus instances](prometheus.md#external-prometheus-instances) to use this endpoint. +NOTE: **Note:** +In GitLab versions 13.1 and greater, you can configure +[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances) +to use this endpoint. ## Setting up generic alerts -To set up the generic alerts integration: +To obtain credentials for setting up a generic alerts integration: -1. Navigate to **Settings > Integrations** in a project. -1. Click on **Alerts endpoint**. -1. Toggle the **Active** alert setting. The `URL` and `Authorization Key` for the webhook configuration can be found there. +- Sign in to GitLab as a user with maintainer [permissions](../../permissions.md) for a project. +- Navigate to the **Operations** page for your project, depending on your installed version of GitLab: + - *In GitLab versions 13.1 and greater,* navigate to **{settings}** **Settings > Operations** in your project. + - *In GitLab versions prior to 13.1,* navigate to **{settings}** **Settings > Integrations** in your project. GitLab will display a banner encouraging you to enable the Alerts endpoint in **{settings}** **Settings > Operations** instead. +- Click **Alerts endpoint**. +- Toggle the **Active** alert setting to display the **URL** and **Authorization Key** for the webhook configuration. ## Customizing the payload @@ -44,6 +49,14 @@ You can customize the payload by sending the following parameters. All fields ot | `severity` | String | The severity of the alert. Must be one of `critical`, `high`, `medium`, `low`, `info`, `unknown`. Default is `critical`. | | `fingerprint` | String or Array | The unique identifier of the alert. This can be used to group occurrences of the same alert. | +You can also add custom fields to the alert's payload. The values of extra parameters +are not limited to primitive types, such as strings or numbers, but can be a nested +JSON object. For example: + +```json +{ "foo": { "bar": { "baz": 42 } } } +``` + TIP: **Payload size:** Ensure your requests are smaller than the [payload application limits](../../../administration/instance_limits.md#generic-alert-json-payloads). @@ -70,6 +83,42 @@ Example payload: "monitoring_tool": "value", "hosts": "value", "severity": "high", - "fingerprint": "d19381d4e8ebca87b55cda6e8eee7385" + "fingerprint": "d19381d4e8ebca87b55cda6e8eee7385", + "foo": { + "bar": { + "baz": 42 + } + } } ``` + +## Triggering test alerts + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab Core in 13.2. + +After a [project maintainer or owner](#setting-up-generic-alerts) +[configures generic alerts](#setting-up-generic-alerts), you can trigger a +test alert to confirm your integration works properly. + +1. Sign in as a user with Developer or greater [permissions](../../../user/permissions.md). +1. Navigate to **{settings}** **Settings > Operations** in your project. +1. Click **Alerts endpoint** to expand the section. +1. Enter a sample payload in **Alert test payload** (valid JSON is required). +1. Click **Test alert payload**. + +GitLab displays an error or success message, depending on the outcome of your test. + +## Automatic grouping of identical alerts **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214557) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +In GitLab versions 13.2 and greater, GitLab groups alerts based on their payload. +When an incoming alert contains the same payload as another alert (excluding the +`start_time` and `hosts` attributes), GitLab groups these alerts together and +displays a counter on the +[Alert Management List](../operations/alert_management.md#alert-management-list) +and details pages. + +If the existing alert is already `resolved`, then a new alert will be created instead. + +![Alert Management List](../operations/img/alert_list_v13_1.png) diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index f0977a4ea76..416996fb629 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -14,7 +14,7 @@ and is automatically configured on [GitHub import](../../../integration/github.m ### Complete these steps on GitHub -This integration requires a [GitHub API token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) +This integration requires a [GitHub API token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) with `repo:status` access granted: 1. Go to your "Personal access tokens" page at diff --git a/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png b/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png new file mode 100644 index 00000000000..5d530a80421 Binary files /dev/null and b/doc/user/project/integrations/img/actions_menu_create_new_dashboard_v13_2.png differ diff --git a/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png b/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png new file mode 100644 index 00000000000..c3a391b06c7 Binary files /dev/null and b/doc/user/project/integrations/img/heatmap_chart_too_much_data_v_13_2.png differ diff --git a/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png b/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png new file mode 100644 index 00000000000..0cf58433b25 Binary files /dev/null and b/doc/user/project/integrations/img/jira/open_jira_issues_list_v13.2.png differ diff --git a/doc/user/project/integrations/img/jira_service_page_v12_2.png b/doc/user/project/integrations/img/jira_service_page_v12_2.png deleted file mode 100644 index ba7dad9b438..00000000000 Binary files a/doc/user/project/integrations/img/jira_service_page_v12_2.png and /dev/null differ diff --git a/doc/user/project/integrations/img/metrics_settings_button_v13_2.png b/doc/user/project/integrations/img/metrics_settings_button_v13_2.png new file mode 100644 index 00000000000..d649f77eded Binary files /dev/null and b/doc/user/project/integrations/img/metrics_settings_button_v13_2.png differ diff --git a/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png b/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png new file mode 100644 index 00000000000..b6ec08f492d Binary files /dev/null and b/doc/user/project/integrations/img/prometheus_manual_configuration_v13_2.png differ diff --git a/doc/user/project/integrations/img/prometheus_service_configuration.png b/doc/user/project/integrations/img/prometheus_service_configuration.png deleted file mode 100644 index a38d1bce197..00000000000 Binary files a/doc/user/project/integrations/img/prometheus_service_configuration.png and /dev/null differ diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 442f3229de2..541c65041ad 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -55,29 +55,41 @@ In order to enable the Jira service in GitLab, you need to first configure the p > **Notes:** > -> - The currently supported Jira versions are `v6.x, v7.x, v8.x` . GitLab 7.8 or -> higher is required. -> - GitLab 8.14 introduced a new way to integrate with Jira which greatly simplified -> the configuration options you have to enter. If you are using an older version, -> [follow this documentation](https://gitlab.com/gitlab-org/gitlab/blob/8-13-stable-ee/doc/project_services/jira.md). +> - The supported Jira versions are `v6.x`, `v7.x`, and `v8.x`. > - In order to support Oracle's Access Manager, GitLab will send additional cookies > to enable Basic Auth. The cookie being added to each request is `OBBasicAuth` with > a value of `fromDialog`. To enable the Jira integration in a project, navigate to the -[Integrations page](overview.md#accessing-integrations), click -the **Jira** service, and fill in the required details on the page as described -in the table below. +[Integrations page](overview.md#accessing-integrations) and click +the **Jira** service. + +Select **Enable integration**. + +Select a **Trigger** action. This determines whether a mention of a Jira issue in GitLab commits, merge requests, or both, should link the Jira issue back to that source commit/MR and transition the Jira issue, if indicated. + +To include a comment on the Jira issue when the above referene is made in GitLab, check **Enable comments**. + +Enter the further details on the page as described in the following table. | Field | Description | | ----- | ----------- | | `Web URL` | The base URL to the Jira instance web interface which is being linked to this GitLab project. E.g., `https://jira.example.com`. | | `Jira API URL` | The base URL to the Jira instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. Leave this field blank (or use the same value of `Web URL`) if using **Jira Cloud**. | -| `Username/Email` | Created when [configuring Jira step](#configuring-jira). Use `username` for **Jira Server** or `email` for **Jira Cloud**. | -| `Password/API token` |Created in [configuring Jira step](#configuring-jira). Use `password` for **Jira Server** or `API token` for **Jira Cloud**. | -| `Transition ID` | This is the ID of a transition that moves issues to the desired state. It is possible to insert transition ids separated by `,` or `;` which means the issue will be moved to each state after another using the given order. **Closing Jira issues via commits or Merge Requests won't work if you don't set the ID correctly.** | +| `Username or Email` | Created in [configuring Jira](#configuring-jira) step. Use `username` for **Jira Server** or `email` for **Jira Cloud**. | +| `Password/API token` |Created in [configuring Jira](#configuring-jira) step. Use `password` for **Jira Server** or `API token` for **Jira Cloud**. | +| `Transition ID` | Required for closing Jira issues via commits or merge requests. This is the ID of a transition in Jira that moves issues to a desired state. (See [Obtaining a transition ID](#obtaining-a-transition-id).) If you insert multiple transition IDs separated by `,` or `;`, the issue is moved to each state, one after another, using the given order. | + +To enable users to view Jira issues inside GitLab, select **Enable Jira issues** and enter a project key. **(PREMIUM)** + +CAUTION: **Caution:** +If you enable Jira issues with the setting above, all users that have access to this GitLab project will be able to view all issues from the specified Jira project. + +When you have configured all settings, click **Test settings and save changes**. -### Obtaining a transition ID +Your GitLab project can now interact with all Jira projects in your instance and the project now displays a Jira link that opens the Jira project. + +#### Obtaining a transition ID In the most recent Jira user interface, you can no longer see transition IDs in the workflow administration UI. You can get the ID you need in either of the following ways: @@ -90,19 +102,11 @@ administration UI. You can get the ID you need in either of the following ways: Note that the transition ID may vary between workflows (e.g., bug vs. story), even if the status you are changing to is the same. -After saving the configuration, your GitLab project will be able to interact -with all Jira projects in your Jira instance and you'll see the Jira link on the GitLab project pages that takes you to the appropriate Jira project. - -![Jira service page](img/jira_service_page_v12_2.png) - -### Disabling comments on Jira issues - -When you reference a Jira issue, it will always link back to the source commit/MR in GitLab, however, you can control whether GitLab will also cross-post a comment to the Jira issue. That functionality is enabled by default. +#### Disabling comments on Jira issues -To disable the automated commenting on Jira issues: +You can continue to have GitLab cross-link a source commit/MR with a Jira issue while disabling the comment added to the issue. -1. Open the [Integrations page](overview.md#accessing-integrations) and select **Jira**. -1. In the **Event Action** section, uncheck **Comment**. +See the [Configuring GitLab](#configuring-gitlab) section and uncheck the **Enable comments** setting. ## Jira issues @@ -111,7 +115,7 @@ By now you should have [configured Jira](#configuring-jira) and enabled the you should be able to reference and close Jira issues by just mentioning their ID in GitLab commits and merge requests. -### Referencing Jira Issues +### Reference Jira issues When GitLab project has Jira issue tracker configured and enabled, mentioning Jira issue in GitLab will automatically add a comment in Jira issue with the @@ -138,7 +142,7 @@ For example, the following commit will reference the Jira issue with `PROJECT-1` git commit -m "PROJECT-1 Fix spelling and grammar" ``` -### Closing Jira Issues +### Close Jira issues Jira issues can be closed directly from GitLab by using trigger words in commits and merge requests. When a commit which contains the trigger word @@ -162,8 +166,6 @@ where `PROJECT-1` is the ID of the Jira issue. > [project settings](img/jira_project_settings.png). > - The Jira issue will not be transitioned if it has a resolution. -### Jira issue closing example - Let's consider the following example: 1. For the project named `PROJECT` in Jira, we implemented a new feature @@ -185,6 +187,45 @@ with a link to the commit that resolved the issue. ![The GitLab integration closes Jira issue](img/jira_service_close_issue.png) +### View Jira issues **(PREMIUM)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3622) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +You can browse and search issues from a selected Jira project directly in GitLab. This requires [configuration](#configuring-gitlab) in GitLab by an administrator. + +![Jira issues integration enabled](img/jira/open_jira_issues_list_v13.2.png) + +From the **Jira Issues** menu, click **Issues List**. The issue list defaults to sort by **Created date**, with the newest issues listed at the top. You can change this to **Last updated**. + +Issues are grouped into tabs based on their [Jira status](https://confluence.atlassian.com/adminjiraserver070/defining-status-field-values-749382903.html). + +- The **Open** tab displays all issues with a Jira status in any category other than Done. +- The **Closed** tab displays all issues with a Jira status categorized as Done. +- The **All** tab displays all issues of any status. + +Click an issue title to open its original Jira issue page for full details. + +#### Search and filter the issues list + +To refine the list of issues, use the search bar to search for any text +contained in an issue summary (title) or description. + +You can also filter by labels, status, reporter, and assignee using URL parameters. +Enhancements to be able to use these through the user interface are [planned](https://gitlab.com/groups/gitlab-org/-/epics/3622). + +- To filter issues by `labels`, specify one or more labels as part of the `labels[]` +parameter in the URL. When using multiple labels, only issues that contain all specified +labels are listed. `/-/integrations/jira/issues?labels[]=backend&labels[]=feature&labels[]=QA` + +- To filter issues by `status`, specify the `status` parameter in the URL. +`/-/integrations/jira/issues?status=In Progress` + +- To filter issues by `reporter`, specify a reporter's Jira display name for the +`author_username` parameter in the URL. `/-/integrations/jira/issues?author_username=John Smith` + +- To filter issues by `assignee`, specify their Jira display name for the +`assignee_username` parameter in the URL. `/-/integrations/jira/issues?assignee_username=John Smith` + ## Troubleshooting If these features do not work as expected, it is likely due to a problem with the way the integration settings were configured. diff --git a/doc/user/project/integrations/jira_cloud_configuration.md b/doc/user/project/integrations/jira_cloud_configuration.md index 619c94b282b..c7157b6bd0e 100644 --- a/doc/user/project/integrations/jira_cloud_configuration.md +++ b/doc/user/project/integrations/jira_cloud_configuration.md @@ -5,7 +5,7 @@ below to create one: 1. Log in to [`id.atlassian.com`](https://id.atlassian.com/manage-profile/security/api-tokens) with your email address. - NOTE: **Note** + NOTE: **Note:** It is important that the user associated with this email address has *write* access to projects in Jira. diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md index 1efd0ff3944..c8278a0f083 100644 --- a/doc/user/project/integrations/jira_server_configuration.md +++ b/doc/user/project/integrations/jira_server_configuration.md @@ -6,7 +6,7 @@ need to integrate with GitLab. As an example, we'll create a user named `gitlab` and add it to a new group named `gitlab-developers`. -NOTE: **Note** +NOTE: **Note:** It is important that the Jira user created for the integration is given 'write' access to your Jira projects. This is covered in the process below. diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 88668ab6c7d..79c55e2d140 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -14,8 +14,6 @@ want to configure. ![Integrations list](img/project_services.png) -Below, you will find a list of the currently supported ones accompanied with comprehensive documentation. - ## Integrations listing Click on the service links to see further configuration instructions and details. @@ -28,6 +26,7 @@ Click on the service links to see further configuration instructions and details | Buildkite | Continuous integration and deployments | Yes | | [Bugzilla](bugzilla.md) | Bugzilla issue tracker | No | | Campfire | Simple web-based real-time group chat | No | +| [Confluence](../../../api/services.md#confluence-service) | Replaces the link to the internal wiki with a link to a Confluence Cloud Workspace | No | | Custom Issue Tracker | Custom issue tracker | No | | [Discord Notifications](discord_notifications.md) | Receive event notifications in Discord | No | | Drone CI | Continuous Integration platform built on Docker, written in Go | Yes | @@ -68,16 +67,16 @@ supported by `push_hooks` and `tag_push_hooks` events won't be executed. The number of branches or tags supported can be changed via [`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). -## Services templates +## Service templates -Services templates is a way to set some predefined values in the Service of -your liking which will then be pre-filled on each project's Service. +Service templates are a way to set predefined values for an integration across +all new projects on the instance. -Read more about [Services templates in this document](services_templates.md). +Read more about [Service templates in this document](services_templates.md). ## Troubleshooting integrations -Some integrations use service hooks for integration with external applications. To confirm which ones use service hooks, see the [integrations listing](#integrations-listing). GitLab stores details of service hook requests made within the last 2 days. To view details of the requests, go to the service's configuration page. +Some integrations use service hooks for integration with external applications. To confirm which ones use service hooks, see the [integrations listing](#integrations-listing) above. GitLab stores details of service hook requests made within the last 2 days. To view details of the requests, go to that integration's configuration page. The **Recent Deliveries** section lists the details of each request made within the last 2 days: @@ -88,17 +87,17 @@ The **Recent Deliveries** section lists the details of each request made within - Relative time in which the request was made To view more information about the request's execution, click the respective **View details** link. -On the details page, you can see the data sent by GitLab (request headers and body) and the data received by GitLab (response headers and body). +On the details page, you can see the request headers and body sent and received by GitLab. -From this page, you can repeat delivery with the same data by clicking **Resend Request**. +To repeat a delivery using the same data, click **Resend Request**. ![Recent deliveries](img/webhook_logs.png) ### Uninitialized repositories Some integrations fail with an error `Test Failed. Save Anyway` when you attempt to set them up on -uninitialized repositories. This is because the default service test uses push data to build the -payload for the test request, and it fails, because there are no push events for the project. +uninitialized repositories. Some integrations use push data to build the test payload, +and this error occurs when no push events exist in the project yet. To resolve this error, initialize the repository by pushing a test file to the project and set up the integration again. diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 0d17ff51372..f1567208a8f 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -19,7 +19,8 @@ There are two ways to set up Prometheus integration, depending on where your app - For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes). - For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus). -Once enabled, GitLab will automatically detect metrics from known services in the [metric library](#monitoring-cicd-environments). You can also [add your own metrics](#adding-custom-metrics). +Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics) and create +[custom dashboards](../../../operations/metrics/dashboards/index.md). ## Enabling Prometheus Integration @@ -32,11 +33,10 @@ GitLab can seamlessly deploy and manage Prometheus on a [connected Kubernetes cl #### Requirements - A [connected Kubernetes cluster](../clusters/index.md) -- Helm Tiller [installed by GitLab](../clusters/index.md#installing-applications) #### Getting started -Once you have a connected Kubernetes cluster with Helm installed, deploying a managed Prometheus is as easy as a single click. +Once you have a connected Kubernetes cluster, deploying a managed Prometheus is as easy as a single click. 1. Go to the **Operations > Kubernetes** page to view your connected clusters 1. Select the cluster you would like to deploy Prometheus to @@ -44,53 +44,6 @@ Once you have a connected Kubernetes cluster with Helm installed, deploying a ma ![Managed Prometheus Deploy](img/prometheus_deploy.png) -#### Getting metrics to display on the Metrics Dashboard - -After completing the steps above, you will also need deployments in order to view the -**Operations > Metrics** page. Setting up [Auto DevOps](../../../topics/autodevops/index.md) -will help you to quickly create a deployment: - -1. Navigate to your project's **Operations > Kubernetes** page, and ensure that, - in addition to "Prometheus" and "Helm Tiller", you also have "Runner" and "Ingress" - installed. Once "Ingress" is installed, copy its endpoint. -1. Navigate to your project's **Settings > CI/CD** page. In the Auto DevOps section, - select a deployment strategy and save your changes. -1. On the same page, in the Variables section, add a variable named `KUBE_INGRESS_BASE_DOMAIN` - with the value of the Ingress endpoint you have copied in the previous step. Leave the type - as "Variable". -1. Navigate to your project's **CI/CD > Pipelines** page, and run a pipeline on any branch. -1. When the pipeline has run successfully, graphs will be available on the **Operations > Metrics** page. - -![Monitoring Dashboard](img/prometheus_monitoring_dashboard_v13_1.png) - -#### Using the Metrics Dashboard - -##### Select an environment - -The **Environment** dropdown box above the dashboard displays the list of all [environments](#monitoring-cicd-environments). -It enables you to search as you type through all environments and select the one you're looking for. - -![Monitoring Dashboard Environments](img/prometheus_dashboard_environments_v12_8.png) - -##### Select a dashboard - -The **dashboard** dropdown box above the dashboard displays the list of all dashboards available for the project. -It enables you to search as you type through all dashboards and select the one you're looking for. - -![Monitoring Dashboard select](img/prometheus_dashboard_select_v_13_0.png) - -##### Mark a dashboard as favorite - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0. - -When viewing a dashboard, click the empty **Star dashboard** **{star-o}** button to mark a -dashboard as a favorite. Starred dashboards display a solid star **{star}** button, -and appear at the top of the dashboard select list. - -To remove dashboard from the favorites list, click the solid **Unstar Dashboard** **{star}** button. - -![Monitoring Dashboard favorite state toggle](img/toggle_metrics_user_starred_dashboard_v13_0.png) - #### About managed Prometheus deployments Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). @@ -128,14 +81,24 @@ Installing and configuring Prometheus to monitor applications is fairly straight The actual configuration of Prometheus integration within GitLab is very simple. All you will need is the domain name or IP address of the Prometheus server you'd like -to integrate with. - -1. Navigate to the [Integrations page](overview.md#accessing-integrations). +to integrate with. If the Prometheus resource is secured with Google's Identity-Aware Proxy (IAP), +additional information like Client ID and Service Account credentials can be passed which +GitLab can use to access the resource. More information about authentication from a +service account can be found at Google's documentation for +[Authenticating from a service account](https://cloud.google.com/iap/docs/authentication-howto#authenticating_from_a_service_account). + +1. Navigate to the [Integrations page](overview.md#accessing-integrations) at + **{settings}** **Settings > Integrations**. 1. Click the **Prometheus** service. -1. Provide the domain name or IP address of your server, for example `http://prometheus.example.com/` or `http://192.0.2.1/`. +1. For **API URL**, provide the domain name or IP address of your server, such as + `http://prometheus.example.com/` or `http://192.0.2.1/`. +1. (Optional) In **Google IAP Audience Client ID**, provide the Client ID of the + Prometheus OAuth Client secured with Google IAP. +1. (Optional) In **Google IAP Service Account JSON**, provide the contents of the + Service Account credentials file that is authorized to access the Prometheus resource. 1. Click **Save changes**. -![Configure Prometheus Service](img/prometheus_service_configuration.png) +![Configure Prometheus Service](img/prometheus_manual_configuration_v13_2.png) #### Thanos configuration in GitLab @@ -158,7 +121,7 @@ one of them will be used: [Prometheus manual configuration](#manual-configuration-of-prometheus) and a [managed Prometheus on Kubernetes](#managed-prometheus-on-kubernetes), the manual configuration takes precedence and is used to run queries from - [dashboards](#defining-custom-dashboards-per-project) and [custom metrics](#adding-custom-metrics). + [dashboards](../../../operations/metrics/dashboards/index.md#defining-custom-dashboards-per-project) and [custom metrics](../../../operations/metrics/index.md#adding-custom-metrics). - If you have managed Prometheus applications installed on Kubernetes clusters at **different** levels (project, group, instance), the order of precedence is described in [Cluster precedence](../../instance/clusters/index.md#cluster-precedence). @@ -166,884 +129,6 @@ one of them will be used: clusters at the **same** level, the Prometheus application of a cluster with a matching [environment scope](../../../ci/environments/index.md#scoping-environments-with-specs) is used. -## Monitoring CI/CD Environments - -Once configured, GitLab will attempt to retrieve performance metrics for any -environment which has had a successful deployment. - -GitLab will automatically scan the Prometheus server for metrics from known servers like Kubernetes and NGINX, and attempt to identify individual environments. The supported metrics and scan process is detailed in our [Prometheus Metrics Library documentation](prometheus_library/index.md). - -You can view the performance dashboard for an environment by [clicking on the monitoring button](../../../ci/environments/index.md#monitoring-environments). - -### Adding custom metrics - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28527) to [GitLab Core](https://about.gitlab.com/pricing/) 12.10. - -Custom metrics can be monitored by adding them on the monitoring dashboard page. Once saved, they will be displayed on the environment performance dashboard provided that either: - -- A [connected Kubernetes cluster](../clusters/add_remove_clusters.md) with the environment scope of `*` is used and [Prometheus installed on the cluster](#enabling-prometheus-integration) -- Prometheus is [manually configured](#manual-configuration-of-prometheus). - -![Add New Metric](img/prometheus_add_metric.png) - -A few fields are required: - -- **Name**: Chart title -- **Type**: Type of metric. Metrics of the same type will be shown together. -- **Query**: Valid [PromQL query](https://prometheus.io/docs/prometheus/latest/querying/basics/). -- **Y-axis label**: Y axis title to display on the dashboard. -- **Unit label**: Query units, for example `req / sec`. Shown next to the value. - -Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**, and **Y-axis label** match between metrics. For example, a metric with **Name** `Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display on the same chart as a second metric with the same values. A **Legend label** is suggested if this feature is used. - -#### Query Variables - -##### Predefined variables - -GitLab supports a limited set of [CI variables](../../../ci/variables/README.md) in the Prometheus query. This is particularly useful for identifying a specific environment, for example with `ci_environment_slug`. The supported variables are: - -- `ci_environment_slug` -- `kube_namespace` -- `ci_project_name` -- `ci_project_namespace` -- `ci_project_path` -- `ci_environment_name` -- `__range` - -NOTE: **Note:** -Variables for Prometheus queries must be lowercase. - -###### __range - -The `__range` variable is useful in Prometheus -[range vector selectors](https://prometheus.io/docs/prometheus/latest/querying/basics/#range-vector-selectors). -Its value is the total number of seconds in the dashboard's time range. -For example, if the dashboard time range is set to 8 hours, the value of -`__range` is `28800s`. - -##### User-defined variables - -[Variables can be defined](#templating-templating-properties) in a custom dashboard YAML file. - -##### Using variables - -Variables can be specified using double curly braces, such as `"{{ci_environment_slug}}"` ([added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20793) in GitLab 12.7). - -Support for the `"%{ci_environment_slug}"` format was -[removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31581) in GitLab 13.0. -Queries that continue to use the old format will show no data. - -#### Query Variables from URL - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214500) in GitLab 13.0. - -GitLab supports setting custom variables through URL parameters. Surround the variable -name with double curly braces (`{{example}}`) to interpolate the variable in a query: - -```plaintext -avg(sum(container_memory_usage_bytes{container_name!="{{pod}}"}) by (job)) without (job) /1024/1024/1024' -``` - -The URL for this query would be: - -```plaintext -http://gitlab.com///-/environments//metrics?dashboard=.gitlab%2Fdashboards%2Fcustom.yml&pod=POD -``` - -#### Editing additional metrics from the dashboard - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9. - -You can edit existing additional custom metrics by clicking the **{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**. - -![Edit metric](img/prometheus_dashboard_edit_metric_link_v_12_9.png) - -### Defining custom dashboards per project - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/59974) in GitLab 12.1. - -By default, all projects include a GitLab-defined Prometheus dashboard, which -includes a few key metrics, but you can also define your own custom dashboards. - -You may create a new file from scratch or duplicate a GitLab-defined Prometheus -dashboard. - -NOTE: **Note:** -The metrics as defined below do not support alerts, unlike -[custom metrics](#adding-custom-metrics). - -#### Adding a new dashboard to your project - -You can configure a custom dashboard by adding a new YAML file into your project's -`.gitlab/dashboards/` directory. In order for the dashboards to be displayed on -the project's **Operations > Metrics** page, the files must have a `.yml` -extension and should be present in the project's **default** branch. - -For example: - -1. Create `.gitlab/dashboards/prom_alerts.yml` under your repository's root - directory with the following contents: - - ```yaml - dashboard: 'Dashboard Title' - panel_groups: - - group: 'Group Title' - panels: - - type: area-chart - title: "Chart Title" - y_label: "Y-Axis" - y_axis: - format: number - precision: 0 - metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Instance: {{instance}}, method: {{method}}" - unit: "count" - ``` - - The above sample dashboard would display a single area chart. Each file should - define the layout of the dashboard and the Prometheus queries used to populate - data. - -1. Save the file, commit, and push to your repository. The file must be present in your **default** branch. -1. Navigate to your project's **Operations > Metrics** and choose the custom - dashboard from the dropdown. - -NOTE: **Note:** -Configuration files nested under subdirectories of `.gitlab/dashboards` are not -supported and will not be available in the UI. - -#### Duplicating a GitLab-defined dashboard - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37238) in GitLab 12.7. -> - From [GitLab 12.8 onwards](https://gitlab.com/gitlab-org/gitlab/-/issues/39505), custom metrics are also duplicated when you duplicate a dashboard. - -You can save a complete copy of a GitLab defined dashboard along with all custom metrics added to it. -Resulting `.yml` file can be customized and adapted to your project. -You can decide to save the dashboard `.yml` file in the project's **default** branch or in a -new branch. - -1. Click **Duplicate dashboard** in the dashboard dropdown. - - NOTE: **Note:** - You can duplicate only GitLab-defined dashboards. - -1. Enter the file name and other information, such as the new commit's message, and click **Duplicate**. - -If you select your **default** branch, the new dashboard becomes immediately available. -If you select another branch, this branch should be merged to your **default** branch first. - -#### Dashboard YAML syntax validation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33202) in GitLab 13.1. - -To confirm your dashboard definition contains valid YAML syntax: - -1. Navigate to **{doc-text}** **Repository > Files**. -1. Navigate to your dashboard file in your repository. -1. Review the information pane about the file, displayed above the file contents. - -Files with valid syntax display **Metrics Dashboard YAML definition is valid**, -and files with invalid syntax display **Metrics Dashboard YAML definition is invalid**. - -![Metrics Dashboard_YAML_syntax_validation](img/prometheus_dashboard_yaml_validation_v13_1.png) - -When **Metrics Dashboard YAML definition is invalid** at least one of the following messages is displayed: - -1. `dashboard: can't be blank` [learn more](#dashboard-top-level-properties) -1. `panel_groups: can't be blank` [learn more](#dashboard-top-level-properties) -1. `group: can't be blank` [learn more](#panel-group-panel_groups-properties) -1. `panels: can't be blank` [learn more](#panel-group-panel_groups-properties) -1. `metrics: can't be blank` [learn more](#panel-panels-properties) -1. `title: can't be blank` [learn more](#panel-panels-properties) -1. `query: can't be blank` [learn more](#metrics-metrics-properties) -1. `query_range: can't be blank` [learn more](#metrics-metrics-properties) -1. `unit: can't be blank` [learn more](#metrics-metrics-properties) -1. `YAML syntax: The parsed YAML is too big` - - This is displayed when the YAML file is larger than 1 MB. - -1. `YAML syntax: Invalid configuration format` - - This is displayed when the YAML file is empty or does not contain valid YAML. - -Metrics Dashboard YAML definition validation information is also available as a [GraphQL API field](../../../api/graphql/reference/index.md#metricsdashboard) - -#### Dashboard YAML properties - -Dashboards have several components: - -- Templating variables. -- Panel groups, which consist of panels. -- Panels, which support one or more metrics. - -The following tables outline the details of expected properties. - -##### **Dashboard (top-level) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `dashboard` | string | yes | Heading for the dashboard. Only one dashboard should be defined per file. | -| `panel_groups` | array | yes | The panel groups which should be on the dashboard. | -| `templating` | hash | no | Top level key under which templating related options can be added. | -| `links` | array | no | Add links to display on the dashboard. | - -##### **Templating (`templating`) properties** - -| Property | Type | Required | Description | -| -------- | ---- | -------- | ----------- | -| `variables` | hash | yes | Variables can be defined here. | - -Read the documentation on [templating](#templating-variables-for-metrics-dashboards). - -##### **Links (`links`) properties** - -| Property | Type | Required | Description | -| -------- | ---- | -------- | ----------- | -| `url` | string | yes | The address of the link. | -| `title` | string | no | Display title for the link. | -| `type` | string | no | Type of the link. Specifies the link type, can be: `grafana` | - -Read the documentation on [links](#add-related-links-to-custom-dashboards). - -##### **Panel group (`panel_groups`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `group` | string | required | Heading for the panel group. | -| `priority` | number | optional, defaults to order in file | Order to appear on the dashboard. Higher number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | -| `panels` | array | required | The panels which should be in the panel group. | - -##### **Panel (`panels`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------- | -| `type` | enum | no, defaults to `area-chart` | Specifies the chart type to use, can be: `area-chart`, `line-chart` or `anomaly-chart`. | -| `title` | string | yes | Heading for the panel. | -| `y_label` | string | no, but highly encouraged | Y-Axis label for the panel. | -| `y_axis` | string | no | Y-Axis configuration for the panel. | -| `max_value` | number | no | Denominator value used for calculating [percentile based results](#percentile-based-results) | -| `weight` | number | no, defaults to order in file | Order to appear within the grouping. Lower number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | -| `metrics` | array | yes | The metrics which should be displayed in the panel. Any number of metrics can be displayed when `type` is `area-chart` or `line-chart`, whereas only 3 can be displayed when `type` is `anomaly-chart`. | -| `links` | array | no | Add links to display on the chart's [context menu](#chart-context-menu). | - -##### **Axis (`panels[].y_axis`) properties** - -| Property | Type | Required | Description | -| ----------- | ------ | ----------------------------- | -------------------------------------------------------------------- | -| `name` | string | no, but highly encouraged | Y-Axis label for the panel. Replaces `y_label` if set. | -| `format` | string | no, defaults to `engineering` | Unit format used. See the [full list of units](prometheus_units.md). | -| `precision` | number | no, defaults to `2` | Number of decimal places to display in the number. | | - -##### **Metrics (`metrics`) properties** - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). | -| `unit` | string | yes | Defines the unit of the query's return data. | -| `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. | -| `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | -| `query_range` | string | yes if `query` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query_range` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | -| `step` | number | no, value is calculated if not defined | Defines query resolution step width in float number of seconds. Metrics on the same panel should use the same `step` value. | - -##### Dynamic labels - -Dynamic labels are useful when multiple time series are returned from a Prometheus query. - -When a static label is used and a query returns multiple time series, then all the legend items will be labeled the same, which makes identifying each time series difficult: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Time Series" - unit: "count" -``` - -This may render a legend like this: - -![repeated legend label chart](img/prometheus_dashboard_repeated_label.png) - -For labels to be more explicit, using variables that reflect time series labels is a good practice. The variables will be replaced by the values of the time series labels when the legend is rendered: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Instance: {{instance}}, method: {{method}}" - unit: "count" -``` - -The resulting rendered legend will look like this: - -![legend with label variables](img/prometheus_dashboard_label_variables.png) - -There is also a shorthand value for dynamic dashboard labels that make use of only one time series label: - -```yaml -metrics: - - id: my_metric_id - query_range: 'http_requests_total' - label: "Method" - unit: "count" -``` - -This works by lowercasing the value of `label` and, if there are more words separated by spaces, replacing those spaces with an underscore (`_`). The transformed value is then checked against the labels of the time series returned by the Prometheus query. If a time series label is found that is equal to the transformed value, then the label value will be used and rendered in the legend like this: - -![legend with label shorthand variable](img/prometheus_dashboard_label_variable_shorthand.png) - -#### Panel types for dashboards - -The below panel types are supported in monitoring dashboards. - -##### Area or Line Chart - -To add an area chart panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - type: area-chart # or line-chart - title: 'Area Chart Title' - y_label: "Y-Axis" - y_axis: - format: number - precision: 0 - metrics: - - id: area_http_requests_total - query_range: 'http_requests_total' - label: "Instance: {{instance}}, Method: {{method}}" - unit: "count" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | no | Type of panel to be rendered. Optional for area panel types | -| query_range | string | required | For area panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - -![area panel chart](img/prometheus_dashboard_area_panel_type_v12_8.png) - -Starting in [version 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/202696), the y-axis values will automatically scale according to the data. Previously, it always started from 0. - -##### Anomaly chart - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16530) in GitLab 12.5. - -To add an anomaly chart panel type to a dashboard, add a panel with *exactly* 3 metrics. - -The first metric represents the current state, and the second and third metrics represent the upper and lower limit respectively: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - type: anomaly-chart - title: "Chart Title" - y_label: "Y-Axis" - metrics: - - id: anomaly_requests_normal - query_range: 'http_requests_total' - label: "# of Requests" - unit: "count" - metrics: - - id: anomaly_requests_upper_limit - query_range: 10000 - label: "Max # of requests" - unit: "count" - metrics: - - id: anomaly_requests_lower_limit - query_range: 2000 - label: "Min # of requests" - unit: "count" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | required | Must be `anomaly-chart` for anomaly panel types | -| query_range | yes | required | For anomaly panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) in every metric. | - -![anomaly panel type](img/prometheus_dashboard_anomaly_panel_type.png) - -##### Bar chart - -To add a bar chart to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group title' - panels: - - type: bar - title: "Http Handlers" - x_label: 'Response Size' - y_axis: - name: "Handlers" - metrics: - - id: prometheus_http_response_size_bytes_bucket - query_range: "sum(increase(prometheus_http_response_size_bytes_bucket[1d])) by (handler)" - unit: 'Bytes' -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `type` | string | yes | Type of panel to be rendered. For bar chart types, set to `bar` | -| `query_range` | yes | yes | For bar chart, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) - -![bar chart panel type](img/prometheus_dashboard_bar_chart_panel_type_v12.10.png) - -##### Column chart - -To add a column panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group title' - panels: - - title: "Column" - type: "column" - metrics: - - id: 1024_memory - query: 'avg(sum(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) by (job)) without (job) / count(avg(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) without (job)) /1024/1024' - unit: MB - label: "Memory Usage" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For column panel types, set to `column` | -| query_range | yes | yes | For column panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - -![anomaly panel type](img/prometheus_dashboard_column_panel_type.png) - -##### Stacked column - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30583) in GitLab 12.8. - -To add a stacked column panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard title' -priority: 1 -panel_groups: -- group: 'Group Title' - priority: 5 - panels: - - type: 'stacked-column' - title: "Stacked column" - y_label: "y label" - x_label: 'x label' - metrics: - - id: memory_1 - query_range: 'memory_query' - label: "memory query 1" - unit: "count" - series_name: 'group 1' - - id: memory_2 - query_range: 'memory_query_2' - label: "memory query 2" - unit: "count" - series_name: 'group 2' - -``` - -![stacked column panel type](img/prometheus_dashboard_stacked_column_panel_type_v12_8.png) - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| `type` | string | yes | Type of panel to be rendered. For stacked column panel types, set to `stacked-column` | -| `query_range` | yes | yes | For stacked column panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - -##### Single Stat - -To add a single stat panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Single Stat" - type: "single-stat" - metrics: - - id: 10 - query: 'max(go_memstats_alloc_bytes{job="prometheus"})' - unit: MB - label: "Total" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For single stat panel types, set to `single-stat` | -| query | string | yes | For single stat panel types, you must use an [instant query](https://prometheus.io/docs/prometheus/latest/querying/api/#instant-queries) | - -![single stat panel type](img/prometheus_dashboard_single_stat_panel_type.png) - -###### Percentile based results - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201946) in GitLab 12.8. - -Query results sometimes need to be represented as a percentage value out of 100. You can use the `max_value` property at the root of the panel definition: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Single Stat" - type: "single-stat" - max_value: 100 - metrics: - - id: 10 - query: 'max(go_memstats_alloc_bytes{job="prometheus"})' - unit: '%' - label: "Total" -``` - -For example, if you have a query value of `53.6`, adding `%` as the unit results in a single stat value of `53.6%`, but if the maximum expected value of the query is `120`, the value would be `44.6%`. Adding the `max_value` causes the correct percentage value to display. - -##### Heatmaps - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30581) in GitLab 12.5. - -To add a heatmap panel type to a dashboard, look at the following sample dashboard file: - -```yaml -dashboard: 'Dashboard Title' -panel_groups: - - group: 'Group Title' - panels: - - title: "Heatmap" - type: "heatmap" - metrics: - - id: 10 - query: 'sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[60m])) by (status_code)' - unit: req/sec - label: "Status code" -``` - -Note the following properties: - -| Property | Type | Required | Description | -| ------ | ------ | ------ | ------ | -| type | string | yes | Type of panel to be rendered. For heatmap panel types, set to `heatmap` | -| query_range | yes | yes | For area panel types, you must use a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries) | - -![heatmap panel type](img/heatmap_panel_type.png) - -### Templating variables for metrics dashboards - -Templating variables can be used to make your metrics dashboard more versatile. - -#### Templating variable types - -`templating` is a top-level key in the -[dashboard YAML](#dashboard-top-level-properties). -Define your variables in the `variables` key, under `templating`. The value of -the `variables` key should be a hash, and each key under `variables` -defines a templating variable on the dashboard, and may contain alphanumeric and underscore characters. - -A variable can be used in a Prometheus query in the same dashboard using the syntax -described [here](#using-variables). - -##### `text` variable type - -CAUTION: **Warning:** -This variable type is an _alpha_ feature, and is subject to change at any time -without prior notice! - -For each `text` variable defined in the dashboard YAML, there will be a free text -box on the dashboard UI, allowing you to enter a value for each variable. - -The `text` variable type supports a simple and a full syntax. - -###### Simple syntax - -This example creates a variable called `variable1`, with a default value -of `default value`: - -```yaml -templating: - variables: - variable1: 'default value' # `text` type variable with `default value` as its default. -``` - -###### Full syntax - -This example creates a variable called `variable1`, with a default value of `default`. -The label for the text box on the UI will be the value of the `label` key: - -```yaml -templating: - variables: - variable1: # The variable name that can be used in queries. - label: 'Variable 1' # (Optional) label that will appear in the UI for this text box. - type: text - options: - default_value: 'default' # (Optional) default value. -``` - -##### `custom` variable type - -CAUTION: **Warning:** -This variable type is an _alpha_ feature, and is subject to change at any time -without prior notice! - -Each `custom` variable defined in the dashboard YAML creates a dropdown -selector on the dashboard UI, allowing you to select a value for each variable. - -The `custom` variable type supports a simple and a full syntax. - -###### Simple syntax - -This example creates a variable called `variable1`, with a default value of `value1`. -The dashboard UI will display a dropdown with `value1`, `value2` and `value3` -as the choices. - -```yaml -templating: - variables: - variable1: ['value1', 'value2', 'value3'] -``` - -###### Full syntax - -This example creates a variable called `variable1`, with a default value of `value_option_2`. -The label for the text box on the UI will be the value of the `label` key. -The dashboard UI will display a dropdown with `Option 1` and `Option 2` -as the choices. - -If you select `Option 1` from the dropdown, the variable will be replaced with `value option 1`. -Similarly, if you select `Option 2`, the variable will be replaced with `value_option_2`: - -```yaml -templating: - variables: - variable1: # The variable name that can be used in queries. - label: 'Variable 1' # (Optional) label that will appear in the UI for this dropdown. - type: custom - options: - values: - - value: 'value option 1' # The value that will replace the variable in queries. - text: 'Option 1' # (Optional) Text that will appear in the UI dropdown. - - value: 'value_option_2' - text: 'Option 2' - default: true # (Optional) This option should be the default value of this variable. -``` - -### Add related links to custom dashboards - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216385) in GitLab 13.1. - -You can embed links to other dashboards or external services in your custom -dashboard by adding **Related links** to your dashboard's YAML file. Related links -open in the same tab as the dashboard. Related links can be displayed in the -following locations on your dashboard: - -- At the top of your dashboard as the top level [`links` dashboard property](#dashboard-top-level-properties). -- In charts context menus as the [`links` property of a panel](#panel-panels-properties). - -Related links can contain the following attributes: - -- `url`: The full URL to the link. Required. -- `title`: A phrase describing the link. Optional. If this attribute is not set, - the full URL is used for the link title. -- `type`: A string declaring the type of link. Optional. If set to `grafana`, the - dashboard's time range values are converted to Grafana's time range format and - appended to the `url`. - -The dashboard's time range is appended to the `url` as URL parameters. - -The following example shows two related links (`GitLab.com` and `GitLab Documentation`) -added to a dashboard: - -![Links UI](img/related_links_v13_1.png) - -#### Links Syntax - -```yaml -links: - - title: GitLab.com - url: https://gitlab.com - - title: GitLab Documentation - url: https://docs.gitlab.com - - title: Public Grafana playground dashboard - url: https://play.grafana.org/d/000000012/grafana-play-home?orgId=1 - type: grafana -``` - -### View and edit the source file of a custom dashboard - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34779) in GitLab 12.5. - -When viewing a custom dashboard of a project, you can view the original -`.yml` file by clicking on the **Edit dashboard** button. - -### Chart Context Menu - -From each of the panels in the dashboard, you can access the context menu by clicking the **{ellipsis_v}** **More actions** dropdown box above the upper right corner of the panel to take actions related to the chart's data. - -![Context Menu](img/panel_context_menu_v13_0.png) - -The options are: - -- [Expand panel](#expand-panel) -- [View logs](#view-logs-ultimate) -- [Download CSV](#downloading-data-as-csv) -- [Copy link to chart](#embedding-gitlab-managed-kubernetes-metrics) -- [Alerts](#setting-up-alerts-for-prometheus-metrics) - -### Dashboard Annotations - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`). -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/215224) in GitLab 13.0. - -You can use **Metrics Dashboard Annotations** to mark any important events on -every metrics dashboard by adding annotations to it. While viewing a dashboard, -annotation entries assigned to the selected time range will be automatically -fetched and displayed on every chart within that dashboard. On mouse hover, each -annotation presents additional details, including the exact time of an event and -its description. - -You can create annotations by making requests to the -[Metrics dashboard annotations API](../../../api/metrics_dashboard_annotations.md) - -![Annotations UI](img/metrics_dashboard_annotations_ui_v13.0.png) - -#### Retention policy - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211433) in GitLab 13.01. - -To avoid excessive storage space consumption by stale annotations, records attached -to time periods older than two weeks are removed daily. This recurring background -job runs at 1:00 a.m. local server time. - -### Expand panel - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3100) in GitLab 13.0. - -To view a larger version of a visualization, expand the panel by clicking the -**{ellipsis_v}** **More actions** icon and selecting **Expand panel**. - -To return to the metrics dashboard, click the **Back** button in your -browser, or pressing the Esc key. - -### View Logs **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/122013) in GitLab 12.8. - -If you have [Logs](../clusters/kubernetes_pod_logs.md) enabled, -you can navigate from the charts in the dashboard to view Logs by -clicking on the context menu in the upper-right corner. - -If you use the **Timeline zoom** function at the bottom of the chart, logs will narrow down to the time range you selected. - -### Timeline zoom and URL sharing - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198910) in GitLab 12.8. - -You can use the **Timeline zoom** function at the bottom of a chart to zoom in -on a date and time of your choice. When you click and drag the sliders to select -a different beginning or end date of data to display, GitLab adds your selected start -and end times to the URL, enabling you to share specific timeframes more easily. - -### Downloading data as CSV - -Data from Prometheus charts on the metrics dashboard can be downloaded as CSV. - -### Setting up alerts for Prometheus metrics - -#### Managed Prometheus instances - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6590) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2 for [custom metrics](#adding-custom-metrics), and 11.3 for [library metrics](prometheus_library/metrics.md). - -For managed Prometheus instances using auto configuration, alerts for metrics [can be configured](#adding-custom-metrics) directly in the performance dashboard. - -To set an alert: - -1. Click on the ellipsis icon in the top right corner of the metric you want to create the alert for. -1. Choose **Alerts** -1. Set threshold and operator. -1. Click **Add** to save and activate the alert. - -![Adding an alert](img/prometheus_alert.png) - -To remove the alert, click back on the alert icon for the desired metric, and click **Delete**. - -#### External Prometheus instances - ->- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8. ->- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10. - -For manually configured Prometheus servers, a notify endpoint is provided to use with Prometheus webhooks. If you have manual configuration enabled, an **Alerts** section is added to **Settings > Integrations > Prometheus**. This contains the *URL* and *Authorization Key*. The **Reset Key** button will invalidate the key and generate a new one. - -![Prometheus service configuration of Alerts](img/prometheus_service_alerts.png) - -To send GitLab alert notifications, copy the *URL* and *Authorization Key* into the [`webhook_configs`](https://prometheus.io/docs/alerting/configuration/#webhook_config) section of your Prometheus Alertmanager configuration: - -```yaml -receivers: - name: gitlab - webhook_configs: - - http_config: - bearer_token: 9e1cbfcd546896a9ea8be557caf13a76 - send_resolved: true - url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json - ... -``` - -In order for GitLab to associate your alerts with an [environment](../../../ci/environments/index.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab. - -NOTE: **Note** -In GitLab versions 13.1 and greater, you can configure your manually configured Prometheus server to use the [Generic alerts integration](generic_alerts.md). - -### Taking action on incidents **(ULTIMATE)** - ->- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11. ->- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue. - -Alerts can be used to trigger actions, like opening an issue automatically (disabled by default since `13.1`). To configure the actions: - -1. Navigate to your project's **Settings > Operations > Incidents**. -1. Enable the option to create issues. -1. Choose the [issue template](../description_templates.md) to create the issue from. -1. Optionally, select whether to send an email notification to the developers of the project. -1. Click **Save changes**. - -Once enabled, an issue will be opened automatically when an alert is triggered which contains values extracted from [alert's payload](https://prometheus.io/docs/alerting/configuration/#webhook_config -): - -- Issue author: `GitLab Alert Bot` -- Issue title: Extract from `annotations/title`, `annotations/summary` or `labels/alertname` -- Alert `Summary`: A list of properties - - `starts_at`: Alert start time via `startsAt` - - `full_query`: Alert query extracted from `generatorURL` - - Optional list of attached annotations extracted from `annotations/*` -- Alert [GFM](../../markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown` - -When GitLab receives a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicating that it was closed automatically by the GitLab Alert bot. - -To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field. - -Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well. - -If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../permissions.md#project-members-permissions) of the project. - ## Determining the performance impact of a merge > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10408) in GitLab 9.2. @@ -1069,174 +154,3 @@ Performance data will be available for the duration it is persisted on the Prometheus server. ![Merge Request with Performance Impact](img/merge_request_performance.png) - -## Embedding metric charts within GitLab Flavored Markdown - -### Embedding GitLab-managed Kubernetes metrics - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/29691) in GitLab 12.2. - -It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm) fields such as issue or merge request descriptions. The maximum number of embedded charts allowed in a GitLab Flavored Markdown field is 100. - -This can be useful if you are sharing an application incident or performance -metrics to others and want to have relevant information directly available. - -NOTE: **Note:** -Requires [Kubernetes](prometheus_library/kubernetes.md) metrics. - -To display metric charts, include a link of the form `https:////-/environments//metrics`: - -![Embedded Metrics Markdown](img/embedded_metrics_markdown_v12_8.png) - -GitLab unfurls the link as an embedded metrics panel: - -![Embedded Metrics Rendered](img/embedded_metrics_rendered_v12_8.png) - -You can also embed a single chart. To get a link to a chart, click the -**{ellipsis_v}** **More actions** menu in the upper right corner of the chart, -and select **Copy link to chart**, as shown in this example: - -![Copy Link To Chart](img/copy_link_to_chart_v12_10.png) - -The following requirements must be met for the metric to unfurl: - -- The `` must correspond to a real environment. -- Prometheus must be monitoring the environment. -- The GitLab instance must be configured to receive data from the environment. -- The user must be allowed access to the monitoring dashboard for the environment ([Reporter or higher](../../permissions.md)). -- The dashboard must have data within the last 8 hours. - - If all of the above are true, then the metric will unfurl as seen below: - -![Embedded Metrics](img/view_embedded_metrics_v12_10.png) - -Metric charts may also be hidden: - -![Show Hide](img/hide_embedded_metrics_v12_10.png) - -You can open the link directly into your browser for a [detailed view of the data](#expand-panel). - -### Embedding metrics in issue templates - -It is also possible to embed either the default dashboard metrics or individual metrics in issue templates. For charts to render side-by-side, links to the entire metrics dashboard or individual metrics should be separated by either a comma or a space. - -![Embedded Metrics in issue templates](img/embed_metrics_issue_template.png) - -### Embedding metrics based on alerts in incident issues - -For [GitLab-managed alerting rules](#setting-up-alerts-for-prometheus-metrics), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after. - -For [manually configured Prometheus instances](#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met: - -- The alert corresponds to an environment managed through GitLab. -- The alert corresponds to a [range query](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries). -- The alert contains the required attributes listed in the chart below. - -| Attributes | Required | Description | -| ---------- | -------- | ----------- | -| `annotations/gitlab_environment_name` | Yes | Name of the GitLab-managed environment corresponding to the alert | -| One of `annotations/title`, `annotations/summary`, `labels/alertname` | Yes | Will be used as the chart title | -| `annotations/gitlab_y_label` | No | Will be used as the chart's y-axis label | - -### Embedding Cluster Health Charts **(ULTIMATE)** - -> [Introduced]() in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. - -[Cluster Health Metrics](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) can also be embedded in [GitLab-flavored Markdown](../../markdown.md). - -To embed a metric chart, include a link to that chart in the form `https:////-/cluster/?` anywhere that GitLab-flavored Markdown is supported. To generate and copy a link to the chart, follow the instructions in the [Cluster Health Metric documentation](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate). - -The following requirements must be met for the metric to unfurl: - -- The `` must correspond to a real cluster. -- Prometheus must be monitoring the cluster. -- The user must be allowed access to the project cluster metrics. -- The dashboards must be reporting data on the [Cluster Health Page](../clusters/index.md#monitoring-your-kubernetes-cluster-ultimate) - - If the above requirements are met, then the metric will unfurl as seen below. - -![Embedded Cluster Metric in issue descriptions](img/prometheus_cluster_health_embed_v12_9.png) - -### Embedding Grafana charts - -Grafana metrics can be embedded in [GitLab Flavored Markdown](../../markdown.md). - -#### Embedding charts via Grafana Rendered Images - -It is possible to embed live [Grafana](https://docs.gitlab.com/omnibus/settings/grafana.html) charts in issues, as a [direct linked rendered image](https://grafana.com/docs/grafana/latest/reference/share_panel/#direct-link-rendered-image). - -The sharing dialog within Grafana provides the link, as highlighted below. - -![Grafana Direct Linked Rendered Image](img/grafana_live_embed.png) - -NOTE: **Note:** -For this embed to display correctly, the Grafana instance must be available to the target user, either as a public dashboard, or on the same network. - -Copy the link and add an image tag as [inline HTML](../../markdown.md#inline-html) in your Markdown. You may tweak the query parameters as required. For instance, removing the `&from=` and `&to=` parameters will give you a live chart. Here is example markup for a live chart from GitLab's public dashboard: - -```html - -``` - -This will render like so: - -![Grafana dashboard embedded preview](img/grafana_embedded.png) - -#### Embedding charts via integration with Grafana HTTP API - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31376) in GitLab 12.5. - -Each project can support integration with one Grafana instance. This configuration allows a user to copy a link to a panel in Grafana, then paste it into a GitLab Markdown field. The chart will be rendered in the GitLab chart format. - -Prerequisites for embedding from a Grafana instance: - -1. The datasource must be a Prometheus instance. -1. The datasource must be proxyable, so the HTTP Access setting should be set to `Server`. - -![HTTP Proxy Access](img/http_proxy_access_v12_5.png) - -##### Setting up the Grafana integration - -1. [Generate an Admin-level API Token in Grafana.](https://grafana.com/docs/grafana/latest/http_api/auth/#create-api-token) -1. In your GitLab project, navigate to **Settings > Operations > Grafana Authentication**. -1. To enable the integration, check the "Active" checkbox. -1. For "Grafana URL", enter the base URL of the Grafana instance. -1. For "API Token", enter the Admin API Token you just generated. -1. Click **Save Changes**. - -##### Generating a link to a chart - -1. In Grafana, navigate to the dashboard you wish to embed a panel from. - ![Grafana Metric Panel](img/grafana_panel_v12_5.png) -1. In the upper-left corner of the page, select a specific value for each variable required for the queries in the chart. - ![Select Query Variables](img/select_query_variables_v12_5.png) -1. In Grafana, click on a panel's title, then click **Share** to open the panel's sharing dialog to the **Link** tab. If you click the _dashboard's_ share panel instead, GitLab will attempt to embed the first supported panel on the dashboard (if available). -1. If your Prometheus queries use Grafana's custom template variables, ensure that the "Template variables" option is toggled to **On**. Of Grafana global template variables, only `$__interval`, `$__from`, and `$__to` are currently supported. Toggle **On** the "Current time range" option to specify the time range of the chart. Otherwise, the default range will be the last 8 hours. - ![Grafana Sharing Dialog](img/grafana_sharing_dialog_v12_5.png) -1. Click **Copy** to copy the URL to the clipboard. -1. In GitLab, paste the URL into a Markdown field and save. The chart will take a few moments to render. - ![GitLab Rendered Grafana Panel](img/rendered_grafana_embed_v12_5.png) - -## Metrics dashboard visibility - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201924) in GitLab 13.0. - -You can set the visibility of the metrics dashboard to **Only Project Members** -or **Everyone With Access**. When set to **Everyone with Access**, the metrics -dashboard is visible to authenticated and non-authenticated users. - -## Troubleshooting - -When troubleshooting issues with a managed Prometheus app, it is often useful to -[view the Prometheus UI](../../../development/prometheus.md#access-the-ui-of-a-prometheus-managed-application-in-kubernetes). - -### "No data found" error on Metrics dashboard page - -If the "No data found" screen continues to appear, it could be due to: - -- No successful deployments have occurred to this environment. -- Prometheus does not have performance data for this environment, or the metrics - are not labeled correctly. To test this, connect to the Prometheus server and - [run a query](prometheus_library/kubernetes.md#metrics-supported), replacing `$CI_ENVIRONMENT_SLUG` - with the name of your environment. -- You may need to re-add the GitLab predefined common metrics. This can be done by running the [import common metrics Rake task](../../../administration/raketasks/maintenance.md#import-common-metrics). diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index b2bc217e8bf..7bebe7b1e65 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -8,7 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22133) in GitLab 11.7. -NOTE: **Note:** NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. +NOTE: **Note:** +NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the built-in Prometheus metrics included with Kubernetes NGINX Ingress controller [version 0.16.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0160) onward. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md index 6ba0a7610f6..326931e9790 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md @@ -8,7 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13438) in GitLab 9.5. -NOTE: **Note:** [NGINX Ingress version 0.16](nginx_ingress.md) and above have built-in Prometheus metrics, which are different than the VTS based metrics. +NOTE: **Note:** +[NGINX Ingress version 0.16](nginx_ingress.md) and above have built-in Prometheus metrics, which are different than the VTS based metrics. GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the included VTS Prometheus metrics exporter in [version 0.9.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#09-beta1) through [0.15.x](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0150). diff --git a/doc/user/project/integrations/prometheus_units.md b/doc/user/project/integrations/prometheus_units.md index 7b216ced1cc..ee4f3ed77d4 100644 --- a/doc/user/project/integrations/prometheus_units.md +++ b/doc/user/project/integrations/prometheus_units.md @@ -1,175 +1,5 @@ --- -stage: Monitor -group: APM -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../../../operations/metrics/dashboards/yaml_number_format.md' --- -# Unit formats reference - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201999) in GitLab 12.9. - -You can select units to format your charts by adding `format` to your -[axis configuration](prometheus.md#dashboard-yaml-properties). - -## Internationalization and localization - -Currently, your [internationalization and localization options](https://en.wikipedia.org/wiki/Internationalization_and_localization) for number formatting are dependent on the system you are using i.e. your OS or browser. - -## Engineering Notation - -For generic or default data, numbers are formatted according to the current locale in [engineering notation](https://en.wikipedia.org/wiki/Engineering_notation). - -While an [engineering notation exists for the web](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat), GitLab uses a version based off the [scientific notation](https://en.wikipedia.org/wiki/Scientific_notation). GitLab formatting acts in accordance with SI prefixes. For example, using GitLab notation, `1500.00` becomes `1.5k` instead of `1.5E3`. Keep this distinction in mind when using the engineering notation for your metrics. - -Formats: `engineering` - -SI prefixes: - -| Name | Symbol | Value | -| ---------- | ------- | -------------------------- | -| `yotta` | Y | 1000000000000000000000000 | -| `zetta` | Z | 1000000000000000000000 | -| `exa` | E | 1000000000000000000 | -| `peta` | P | 1000000000000000 | -| `tera` | T | 1000000000000 | -| `giga` | G | 1000000000 | -| `mega` | M | 1000000 | -| `kilo` | k | 1000 | -| `milli` | m | 0.001 | -| `micro` | μ | 0.000001 | -| `nano` | n | 0.000000001 | -| `pico` | p | 0.000000000001 | -| `femto` | f | 0.000000000000001 | -| `atto` | a | 0.000000000000000001 | -| `zepto` | z | 0.000000000000000000001 | -| `yocto` | y | 0.000000000000000000000001 | - -**Examples:** - -| Data | Displayed | -| --------------------------------- | --------- | -| `0.000000000000000000000008` | 8y | -| `0.000000000000000000008` | 8z | -| `0.000000000000000008` | 8a | -| `0.000000000000008` | 8f | -| `0.000000000008` | 8p | -| `0.000000008` | 8n | -| `0.000008` | 8μ | -| `0.008` | 8m | -| `10` | 10 | -| `1080` | 1.08k | -| `18000` | 18k | -| `18888` | 18.9k | -| `188888` | 189k | -| `18888888` | 18.9M | -| `1888888888` | 1.89G | -| `1888888888888` | 1.89T | -| `1888888888888888` | 1.89P | -| `1888888888888888888` | 1.89E | -| `1888888888888888888888` | 1.89Z | -| `1888888888888888888888888` | 1.89Y | -| `1888888888888888888888888888` | 1.89e+27 | - -## Numbers - -For number data, numbers are formatted according to the current locale. - -Formats: `number` - -**Examples:** - -| Data | Displayed | -| ---------- | --------- | -| `10` | 1 | -| `1000` | 1,000 | -| `1000000` | 1,000,000 | - -## Percentage - -For percentage data, format numbers in the chart with a `%` symbol. - -Formats supported: `percent`, `percentHundred` - -**Examples:** - -| Format | Data | Displayed | -| ---------------- | ----- | --------- | -| `percent` | `0.5` | 50% | -| `percent` | `1` | 100% | -| `percent` | `2` | 200% | -| `percentHundred` | `50` | 50% | -| `percentHundred` | `100` | 100% | -| `percentHundred` | `200` | 200% | - -## Duration - -For time durations, format numbers in the chart with a time unit symbol. - -Formats supported: `milliseconds`, `seconds` - -**Examples:** - -| Format | Data | Displayed | -| -------------- | ------ | --------- | -| `milliseconds` | `10` | 10ms | -| `milliseconds` | `500` | 100ms | -| `milliseconds` | `1000` | 1000ms | -| `seconds` | `10` | 10s | -| `seconds` | `500` | 500s | -| `seconds` | `1000` | 1000s | - -## Digital (Metric) - -Converts a number of bytes using metric prefixes. It scales to -use the unit that's the best fit. - -Formats supported: - -- `decimalBytes` -- `kilobytes` -- `megabytes` -- `gigabytes` -- `terabytes` -- `petabytes` - -**Examples:** - -| Format | Data | Displayed | -| -------------- | --------- | --------- | -| `decimalBytes` | `1` | 1B | -| `decimalBytes` | `1000` | 1kB | -| `decimalBytes` | `1000000` | 1MB | -| `kilobytes` | `1` | 1kB | -| `kilobytes` | `1000` | 1MB | -| `kilobytes` | `1000000` | 1GB | -| `megabytes` | `1` | 1MB | -| `megabytes` | `1000` | 1GB | -| `megabytes` | `1000000` | 1TB | - -## Digital (IEC) - -Converts a number of bytes using binary prefixes. It scales to -use the unit that's the best fit. - -Formats supported: - -- `bytes` -- `kibibytes` -- `mebibytes` -- `gibibytes` -- `tebibytes` -- `pebibytes` - -**Examples:** - -| Format | Data | Displayed | -| ----------- | ------------- | --------- | -| `bytes` | `1` | 1B | -| `bytes` | `1024` | 1KiB | -| `bytes` | `1024 * 1024` | 1MiB | -| `kibibytes` | `1` | 1KiB | -| `kibibytes` | `1024` | 1MiB | -| `kibibytes` | `1024 * 1024` | 1GiB | -| `mebibytes` | `1` | 1MiB | -| `mebibytes` | `1024` | 1GiB | -| `mebibytes` | `1024 * 1024` | 1TiB | +This document was moved to [another location](../../../operations/metrics/dashboards/yaml_number_format.md). diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 9e1cdf0490c..c92ddf38ad2 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -7,7 +7,6 @@ | Field | Description | | ----- | ----------- | - | `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | | `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project. **This is currently not being used and will be removed in a future release.** | diff --git a/doc/user/project/integrations/services_templates.md b/doc/user/project/integrations/services_templates.md index 8a88df88629..bc2bdde2f64 100644 --- a/doc/user/project/integrations/services_templates.md +++ b/doc/user/project/integrations/services_templates.md @@ -1,28 +1,25 @@ -# Services templates +# Service templates -A GitLab administrator can add a service template that sets a default for each -project. After a service template is enabled, it will be applied to **all** -projects that don't have it already enabled and its details will be pre-filled -on the project's Service page. By disabling the template, it will be disabled -for new projects only. +Using a service template, GitLab administrators can provide default values for configuring integrations at the project level. + +When you enable a service template, the defaults are applied to **all** projects that do not +already have the integration enabled or do not otherwise have custom values saved. +The values are pre-filled on each project's configuration page for the applicable integration. + +If you disable the template, these values no longer appear as defaults, while +any values already saved for an integration remain unchanged. ## Enable a service template Navigate to the **Admin Area > Service Templates** and choose the service template you wish to create. -## Services for external issue trackers +## Service for external issue trackers -In the image below you can see how a service template for Redmine would look -like. +The following image shows an example service template for Redmine. ![Redmine service template](img/services_templates_redmine_example.png) ---- - For each project, you will still need to configure the issue tracking URLs by replacing `:issues_tracker_id` in the above screenshot with the ID used -by your external issue tracker. Prior to GitLab v7.8, this ID was configured in -the project settings, and GitLab would automatically update the URL configured -in `gitlab.yml`. This behavior is now deprecated and all issue tracker URLs -must be configured directly within the project's **Integrations** settings. +by your external issue tracker. diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 79fc8eceddf..6c5dc787c5e 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -1,25 +1,45 @@ # Slack Notifications Service -The Slack Notifications Service allows your GitLab project to send events (e.g. issue created) to your existing Slack team as notifications. This requires configurations in both Slack and GitLab. +The Slack Notifications Service allows your GitLab project to send events +(such as issue creation) to your existing Slack team as notifications. Setting up +Slack notifications requires configuration changes for both Slack and GitLab. -> Note: You can also use Slack slash commands to control GitLab inside Slack. This is the separately configured [Slack slash commands](slack_slash_commands.md). +NOTE: **Note:** +You can also use Slack slash commands to control GitLab inside Slack. This is the +separately configured [Slack slash commands](slack_slash_commands.md). -## Slack Configuration +## Slack configuration 1. Sign in to your Slack team and [start a new Incoming WebHooks configuration](https://my.slack.com/services/new/incoming-webhook). -1. Select the Slack channel where notifications will be sent to by default. Click the **Add Incoming WebHooks integration** button to add the configuration. -1. Copy the **Webhook URL**, which we'll use later in the GitLab configuration. +1. Select the Slack channel where notifications will be sent to by default. + Click the **Add Incoming WebHooks integration** button to add the configuration. +1. Copy the **Webhook URL**, which we will use later in the GitLab configuration. -## GitLab Configuration +## GitLab configuration -1. Navigate to the [Integrations page](overview.md#accessing-integrations) in your project's settings, i.e. **Project > Settings > Integrations**. +1. Open your project's page, and navigate to your project's + [Integrations page](overview.md#accessing-integrations) at + **{settings}** **Settings > Integrations**. 1. Select the **Slack notifications** integration to configure it. -1. Ensure that the **Active** toggle is enabled. -1. Check the checkboxes corresponding to the GitLab events you want to send to Slack as a notification. -1. For each event, optionally enter the Slack channel names where you want to send the event, separated by a comma. If left empty, the event will be sent to the default channel that you configured in the Slack Configuration step. **Note:** Usernames and private channels are not supported. To send direct messages, use the Member ID found under user's Slack profile. -1. Paste the **Webhook URL** that you copied from the Slack Configuration step. -1. Optionally customize the Slack bot username that will be sending the notifications. -1. Configure the remaining options and click `Save changes`. +1. Click **Enable integration**. +1. In **Trigger**, select the checkboxes for each type of GitLab event to send to Slack as a + notification. See [Triggers available for Slack notifications](#triggers-available-for-slack-notifications) + for a full list. By default, messages are sent to the channel you configured during + [Slack integration](#slack-configuration). +1. (Optional) To send messages to a different channel, multiple channels, or as a direct message: + - To send messages to channels, enter the Slack channel names, separated by commas. + - To send direct messages, use the Member ID found in the user's Slack profile. + + NOTE: **Note:** + Usernames and private channels are not supported. + +1. In **Webhook**, provide the webhook URL that you copied from the + [Slack integration](#slack-configuration) step. +1. (Optional) In **Username**, provide the username of the Slack bot that sends the notifications. +1. Select the **Notify only broken pipelines** check box to only notify on failures. +1. In the **Branches to be notified** select box, choose which types of branches + to send notifications for. +1. Click **Test settings and save changes**. Your Slack team will now start receiving GitLab event notifications as configured. @@ -43,14 +63,14 @@ The following triggers are available for Slack notifications: ## Troubleshooting -If you're having trouble with the Slack integration not working, then start by +If your Slack integration is not working, start troubleshooting by searching through the [Sidekiq logs](../../../administration/logs.md#sidekiqlog) for errors relating to your Slack service. ### Something went wrong on our end -This is a generic error shown in the GitLab UI and doesn't mean much by itself. -You'll need to look in [the logs](../../../administration/logs.md#productionlog) to find +This is a generic error shown in the GitLab UI and does not mean much by itself. +Review [the logs](../../../administration/logs.md#productionlog) to find an error message and keep troubleshooting from there. ### `certificate verify failed` @@ -83,10 +103,10 @@ result = Net::HTTP.get(URI('https://'));0 result = Net::HTTP.get(URI('https://'));0 ``` -If it's an issue with GitLab not trusting HTTPS connections to itself, then you may simply +If GitLab is not trusting HTTPS connections to itself, then you may need to [add your certificate to GitLab's trusted certificates](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). -If it's an issue with GitLab not trusting connections to Slack, then the GitLab -OpenSSL trust store probably got messed up somehow. Typically this is from overriding -the trust store with `gitlab_rails['env'] = {"SSL_CERT_FILE" => "/path/to/file.pem"}` +If GitLab is not trusting connections to Slack, then the GitLab +OpenSSL trust store is incorrect. Some typical causes: overriding +the trust store with `gitlab_rails['env'] = {"SSL_CERT_FILE" => "/path/to/file.pem"}`, or by accidentally modifying the default CA bundle `/opt/gitlab/embedded/ssl/certs/cacert.pem`. diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index 119a53f5c35..e067ab6071e 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -13,7 +13,6 @@ To enable YouTrack integration in a project: | Field | Description | |:----------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| - | **Description** | Name for the issue tracker (to differentiate between instances, for example). | | **Project URL** | URL to the project in YouTrack which is being linked to this GitLab project. | | **Issues URL** | URL to the issue in YouTrack project that is linked to this GitLab project. Note that the **Issues URL** requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 89b17609698..1831563332f 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -42,8 +42,6 @@ Check all the [GitLab Enterprise features for issue boards](#gitlab-enterprise-f ![GitLab issue boards - Premium](img/issue_boards_premium.png) ---- - ## How it works The Issue Board feature builds on GitLab's existing @@ -98,8 +96,6 @@ If you have the labels "**backend**", "**frontend**", "**staging**", and ![issue card moving](img/issue_board_move_issue_card_list.png) ---- - ### Use cases for multiple issue boards With [multiple issue boards](#multiple-issue-boards), @@ -252,8 +248,6 @@ clicking **View scope**. ![Viewing board configuration](img/issue_board_view_scope.png) ---- - ### Focus mode > - [Introduced]((https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#issue-boards-focus-mode-ees-eep)) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1. @@ -275,8 +269,6 @@ especially in combination with [assignee lists](#assignee-lists-premium). ![issue board summed weights](img/issue_board_summed_weights.png) ---- - ### Group issue boards **(PREMIUM)** > [Introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. @@ -292,8 +284,6 @@ Multiple group issue boards were originally [introduced](https://about.gitlab.co ![Group issue board](img/group_issue_board.png) ---- - ### Assignee lists **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. @@ -313,8 +303,6 @@ To remove an assignee list, just as with a label list, click the trash icon. ![Assignee lists](img/issue_board_assignee_lists.png) ---- - ### Milestone lists **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6469) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.2. @@ -332,8 +320,6 @@ As in other list types, click the trash icon to remove a list. ![Milestone lists](img/issue_board_milestone_lists.png) ---- - ## Work In Progress limits **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11403) in GitLab 12.7 @@ -365,8 +351,6 @@ status. ![Blocked issues](img/issue_boards_blocked_icon_v12_8.png) ---- - ## Actions you can take on an issue board - [Create a new list](#create-a-new-list). @@ -437,8 +421,6 @@ the list by filtering by author, assignee, milestone, and label. ![Bulk adding issues to lists](img/issue_boards_add_issues_modal.png) ---- - ### Remove an issue from a list Removing an issue from a list can be done by clicking the issue card and then @@ -447,8 +429,6 @@ respective label is removed. ![Remove issue from list](img/issue_boards_remove_issue.png) ---- - ### Filter issues You should be able to use the filters on top of your issue board to show only @@ -492,8 +472,6 @@ to another list the label changes and a system not is recorded. ![issue board system notes](img/issue_board_system_notes.png) ---- - ### Drag issues between lists When dragging issues between lists, different behavior occurs depending on the source list and the target list. @@ -518,8 +496,6 @@ To select and move multiple cards: ![Multi-select Issue Cards](img/issue_boards_multi_select_v12_4.png) ---- - ### Issue ordering in a list When visiting a board, issues appear ordered in any list. You're able to change diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index 6cc31a45309..c721bef8f4d 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -31,7 +31,8 @@ git commit -m "this is my commit message. Related to https://gitlab.com/ General**, expand **Visibility, project features, permissions** and enable **Git Large File Storage**. -Design Management requires that projects are using -[hashed storage](../../../administration/repository_storage_types.md#hashed-storage) -(the default storage type since v10.0). +Design Management also requires that projects are using +[hashed storage](../../../administration/raketasks/storage.md#migrate-to-hashed-storage). Since + GitLab 10.0, newly created projects use hashed storage by default. A GitLab admin can verify the storage type of a +project by navigating to **Admin Area > Projects** and then selecting the project in question. +A project can be identified as hashed-stored if its *Gitaly relative path* contains `@hashed`. If the requirements are not met, the **Designs** tab displays a message to the user. @@ -47,6 +49,7 @@ and [PDFs](https://gitlab.com/gitlab-org/gitlab/-/issues/32811) is planned for a ## Limitations - Design uploads are limited to 10 files at a time. +- From GitLab 13.1, Design filenames are limited to 255 characters. - Design Management data [isn't deleted when a project is destroyed](https://gitlab.com/gitlab-org/gitlab/-/issues/13429) yet. - Design Management data [won't be moved](https://gitlab.com/gitlab-org/gitlab/-/issues/13426) @@ -57,20 +60,62 @@ and [PDFs](https://gitlab.com/gitlab-org/gitlab/-/issues/32811) is planned for a - Only the latest version of the designs can be deleted. - Deleted designs cannot be recovered but you can see them on previous designs versions. -## The Design Management page +## GitLab-Figma plugin -Navigate to the **Design Management** page from any issue by clicking the **Designs** tab: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-figma-plugin/-/issues/2) in GitLab 13.2. -![Designs tab](img/design_management_v12_3.png) +Connect your design environment with your source code management in a seamless workflow. The GitLab-Figma plugin makes it quick and easy to collaborate in GitLab by bringing the work of product designers directly from Figma to GitLab Issues as uploaded Designs. + +To use the plugin, install it from the [Figma Directory](https://www.figma.com/community/plugin/860845891704482356) +and connect to GitLab through a personal access token. The details are explained in the [plugin documentation](https://gitlab.com/gitlab-org/gitlab-figma-plugin/-/wikis/home). + +## The Design Management section + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223193) in GitLab 13.2, Designs are displayed directly on the issue description rather than on a separate tab. +> - The new display is deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It cannot be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-displaying-designs-on-the-issue-description-core-only). If disabled, it will move Designs back to the **Designs** tab. + +You can find to the **Design Management** section in the issue description: + +![Designs section](img/design_management_v13_2.png) + +### Enable or disable displaying Designs on the issue description **(CORE ONLY)** + +Displaying Designs on the issue description is under development but ready for +production use. It is deployed behind a feature flag that is **enabled by +default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it for your instance. + +To disable it: + +```ruby +Feature.disable(:design_management_moved) +``` + +To enable it: + +```ruby +Feature.enable(:design_management_moved) +``` + +By disabling this feature, designs will be displayed on the **Designs** tab +instead of directly on the issue description. ## Adding designs -To upload design images, click the **Upload Designs** button and select images to upload. +To upload Design images, drag files from your computer and drop them in the Design Management section, +or click **upload** to select images from your file browser: + +![Designs empty state](img/design_management_upload_v13.2.png) [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34353) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.9, you can drag and drop designs onto the dedicated drop zone to upload them. -![Drag and drop design uploads](img/design_drag_and_drop_uploads_v12_9.png) +![Drag and drop design uploads](img/design_drag_and_drop_uploads_v13_2.png) [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202634) in GitLab 12.10, you can also copy images from your file system and @@ -239,3 +284,13 @@ To disable it: ```ruby Feature.disable(:design_management_reference_filter_gfm_pipeline) ``` + +## Design activity records + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33051) in GitLab 13.1. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/225205) in GitLab 13.2. + +User activity events on designs (creation, deletion, and updates) are tracked by GitLab and +displayed on the [user profile](../../profile/index.md#user-profile), +[group](../../group/index.md#view-group-activity), +and [project](../index.md#project-activity) activity pages. diff --git a/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png b/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png new file mode 100644 index 00000000000..d60f1234b6d Binary files /dev/null and b/doc/user/project/issues/img/design_drag_and_drop_uploads_v13_2.png differ diff --git a/doc/user/project/issues/img/design_management_upload_v13.2.png b/doc/user/project/issues/img/design_management_upload_v13.2.png new file mode 100644 index 00000000000..1d4b10307fc Binary files /dev/null and b/doc/user/project/issues/img/design_management_upload_v13.2.png differ diff --git a/doc/user/project/issues/img/design_management_v13_2.png b/doc/user/project/issues/img/design_management_v13_2.png new file mode 100644 index 00000000000..0a6e2be17ab Binary files /dev/null and b/doc/user/project/issues/img/design_management_v13_2.png differ diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 06a80672769..9113f5344ba 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -175,8 +175,6 @@ requires [GraphQL](../../../api/graphql/index.md) to be enabled. ![Similar issues](img/similar_issues.png) ---- - ### Health status **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36427) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 6d34f91d37f..7f236d04fb6 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -88,7 +88,7 @@ An issue can be assigned to: - Yourself. - Another person. -- [Many people](#multiple-assignees-STARTER). **(STARTER)** +- [Many people](#multiple-assignees-starter). **(STARTER)** The assignee(s) can be changed as often as needed. The idea is that the assignees are responsible for that issue until it's reassigned to someone else to take it from there. @@ -253,7 +253,7 @@ Also: ### Create Merge Request -Create a new branch and [WIP merge request](../merge_requests/work_in_progress_merge_requests.md) +Create a new branch and [**Draft** merge request](../merge_requests/work_in_progress_merge_requests.md) in one action. The branch will be named `issuenumber-title` by default, but you can choose any name, and GitLab will verify that it is not already in use. The merge request will automatically inherit the milestone and labels of the issue, and will be set to diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 08e3164b2eb..babc5030337 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -45,8 +45,7 @@ There are many ways to get to the New Issue form from within a project: ### Elements of the New Issue form -> Ability to add the new issue to an epic [was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) -> in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. +> Ability to add the new issue to an epic [was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. ![New issue from the issues list](img/new_issue_v13_1.png) @@ -76,7 +75,7 @@ create issues for the same project. ![Create issue from group-level issue tracker](img/create_issue_from_group_level_issue_tracker.png) -### New issue via Service Desk **(STARTER)** +### New issue via Service Desk Enable [Service Desk](../service_desk.md) for your project and offer email support. By doing so, when your customer sends a new email, a new issue can be created in diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 406938519b1..9886ef91f16 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -54,7 +54,7 @@ and edit labels. View the project labels list by going to the project and clicking **Issues > Labels**. The list includes all labels that are defined at the project level, as well as all -labels inherited from the parent group. You can filter the list by entering a search +labels inherited from the immediate parent group. You can filter the list by entering a search query at the top and clicking search (**{search}**). To create a new project label: @@ -94,7 +94,7 @@ also be merged. All issues, merge requests, issue board lists, issue board filters, and label subscriptions with the old labels will be assigned to the new group label. -WARNING: **Caution:** +CAUTION: **Caution:** Promoting a label is a permanent action, and cannot be reversed. To promote a project label to a group label: @@ -251,3 +251,16 @@ If you sort by `Priority`, GitLab uses this sort comparison order: Ties are broken arbitrarily. ![Labels sort priority](img/labels_sort_priority.png) + +## Troubleshooting + +### Some label titles end with `_duplicate` + +In specific circumstances it was possible to create labels with duplicate titles in the same +namespace. + +To resolve the duplication, [in GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21384) +and later, some duplicate labels have `_duplicate` appended to their titles. + +You can safely change these labels' titles if you prefer. +For details of the original problem, see [issue 30390](https://gitlab.com/gitlab-org/gitlab/issues/30390). diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 787a74b0065..3eb411aef18 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -128,3 +128,30 @@ If you change your mind before your request is approved, just click the ## Share project with group Alternatively, you can [share a project with an entire group](share_project_with_groups.md) instead of adding users one by one. + +## Remove a member from the project + +Only users with permissions of [Owner](../../permissions.md#group-members-permissions) can manage +project members. + +You can remove a user from the project if the given member has a direct membership in the project. +If membership is inherited from a parent group, then the member can be removed only from the parent +group itself. + +When removing a member, you can decide whether to unassign the user from all issues and merge +requests they are currently assigned or leave the assignments as they are. + +- **Unassigning the removed member** from all issues and merge requests might be helpful when a user + is leaving a private project and you wish to revoke their access to any issues and merge requests + they are assigned. +- **Keeping the issues and merge requests assigned** might be helpful for projects that accept public + contributions where a user doesn't have to be a member to be able to contribute to issues and + merge requests. + +To remove a member from a project: + +1. In a project, go to **{users}** **Members**. +1. Click the **Delete** **{remove}** button next to a project member you want to remove. + A **Remove member** modal appears. +1. (Optional) Select the **Also unassign this user from related issues and merge requests** checkbox. +1. Click **Remove member**. diff --git a/doc/user/project/merge_requests/accessibility_testing.md b/doc/user/project/merge_requests/accessibility_testing.md index 417b85a6082..f3a0aac9ff4 100644 --- a/doc/user/project/merge_requests/accessibility_testing.md +++ b/doc/user/project/merge_requests/accessibility_testing.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- @@ -20,7 +23,10 @@ analyzed to a file called `accessibility`. ## Accessibility Merge Request widget -[Since GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/39425), in addition to the report artifact that is created, GitLab will also show the +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39425) in GitLab 13.0 behind the disabled [feature flag](../../../administration/feature_flags.md) `:accessibility_report_view`. +> - [Feature Flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217372) in GitLab 13.1. + +In addition to the report artifact that is created, GitLab will also show the Accessibility Report in the merge request widget area: ![Accessibility Merge Request Widget](img/accessibility_mr_widget_v13_0.png) diff --git a/doc/user/project/merge_requests/browser_performance_testing.md b/doc/user/project/merge_requests/browser_performance_testing.md index 390d480724d..10457e40e0b 100644 --- a/doc/user/project/merge_requests/browser_performance_testing.md +++ b/doc/user/project/merge_requests/browser_performance_testing.md @@ -1,4 +1,7 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- @@ -7,20 +10,16 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. If your application offers a web interface and you're using -[GitLab CI/CD](../../../ci/README.md), you can quickly determine the performance -impact of pending code changes. +[GitLab CI/CD](../../../ci/README.md), you can quickly determine the rendering performance +impact of pending code changes in the browser. ## Overview GitLab uses [Sitespeed.io](https://www.sitespeed.io), a free and open source -tool, for measuring the performance of web sites. GitLab has built a simple -[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) which outputs -the performance score for each page analyzed in a file called `performance.json`. -The [Sitespeed.io performance score](https://examples.sitespeed.io/6.0/2017-11-23-23-43-35/help.html) -is a composite value based on best practices. - -GitLab can [show the Performance report](#how-browser-performance-testing-works) -in the merge request widget area. +tool, for measuring the rendering performance of web sites. The +[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) that GitLab built outputs +the performance score for each page analyzed in a file called `browser-performance.json` +this data can be shown on Merge Requests. ## Use cases @@ -38,7 +37,7 @@ Consider the following workflow: ## How browser performance testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). +[Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). GitLab then checks this report, compares key performance metrics for each page between the source and target branches, and shows the information in the merge request. @@ -46,12 +45,13 @@ For an example Performance job, see [Configuring Browser Performance Testing](#configuring-browser-performance-testing). NOTE: **Note:** -If the Performance report has no data to compare, such as when you add the -Performance job in your `.gitlab-ci.yml` for the very first time, no information -displays in the merge request widget area. Consecutive merge requests will have data for -comparison, and the Performance report will be shown properly. +If the Browser Performance report has no data to compare, such as when you add the +Browser Performance job in your `.gitlab-ci.yml` for the very first time, +the Browser Performance report widget won't show. It must have run at least +once on the target branch (`master`, for example), before it will display in a +merge request targeting that branch. -![Performance Widget](img/browser_performance_testing.png) +![Browser Performance Widget](img/browser_performance_testing.png) ## Configuring Browser Performance Testing @@ -61,21 +61,7 @@ using Docker-in-Docker. 1. First, set up GitLab Runner with a [Docker-in-Docker build](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). -1. After configuring the Runner, add a new job to `.gitlab-ci.yml` that generates - the expected report. -1. Define the `performance` job according to your version of GitLab: - - - For GitLab 12.4 and later - [include](../../../ci/yaml/README.md#includetemplate) the - [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml) provided as a part of your GitLab installation. - - For GitLab versions earlier than 12.4 - Copy and use the job as defined in the - [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). - - CAUTION: **Caution:** - The job definition provided by the template does not support Kubernetes yet. - For a complete example of a more complex setup that works in Kubernetes, see - [`Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml). - -1. Add the following to your `.gitlab-ci.yml` file: +1. Configure the default Browser Performance Testing CI job as follows in your `.gitlab-ci.yml` file: ```yaml include: @@ -86,24 +72,32 @@ using Docker-in-Docker. URL: https://example.com ``` - CAUTION: **Caution:** - The job definition provided by the template is supported in GitLab 11.5 and later versions. - It also requires GitLab Runner 11.5 or later. For earlier versions, use the - [previous job definitions](#previous-job-definitions). +NOTE: **Note:** +For versions before 12.4, see the information for [older GitLab versions](#gitlab-versions-123-and-older). +If you are using a Kubernetes cluster, use [`template: Jobs/Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml) +instead. The above example creates a `performance` job in your CI/CD pipeline and runs sitespeed.io against the webpage you defined in `URL` to gather key metrics. -The [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance) -is downloaded to save the report as a [Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) -that you can later download and analyze. Due to implementation limitations, we always -take the latest Performance artifact available. -The full HTML sitespeed.io report is saved as an artifact, and if -[GitLab Pages](../pages/index.md) is enabled, it can be viewed directly in your browser. +The example uses a CI/CD template that is included in all GitLab installations since +12.4, but it will not work with Kubernetes clusters. If you are using GitLab 12.3 +or older, you must [add the configuration manually](#gitlab-versions-123-and-older) + +The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance), +and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) +that you can later download and analyze. This implementation always takes the latest +Browser Performance artifact available. If [GitLab Pages](../pages/index.md) is enabled, +you can view the report directly in your browser. + +You can also customize the jobs with environment variables: + +- `SITESPEED_IMAGE`: Configure the Docker image to use for the job (default `sitespeedio/sitespeed.io`), but not the image version. +- `SITESPEED_VERSION`: Configure the version of the Docker image to use for the job (default `13.3.0`). +- `SITESPEED_OPTIONS`: Configure any additional sitespeed.io options as required (default `nil`). Refer to the [sitespeed.io documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) for more details. -You can also customize options by setting the `SITESPEED_OPTIONS` variable. For example, you can override the number of runs sitespeed.io -makes on the given URL: +makes on the given URL, and change the version: ```yaml include: @@ -111,18 +105,11 @@ include: performance: variables: - URL: https://example.com + URL: https://www.sitespeed.io/ + SITESPEED_VERSION: 13.2.0 SITESPEED_OPTIONS: -n 5 ``` -For further customization options for sitespeed.io, including the ability to provide a -list of URLs to test, please see the -[Sitespeed.io Configuration](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) -documentation. - -TIP: **Tip:** -Key metrics are automatically extracted and shown in the merge request widget. - ### Configuring degradation threshold > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27599) in GitLab 13.0. @@ -149,15 +136,12 @@ The above CI YAML configuration is great for testing against static environments be extended for dynamic environments, but a few extra steps are required: 1. The `performance` job should run after the dynamic environment has started. -1. In the `review` job, persist the hostname and upload it as an artifact so - it's available to the `performance` job. The same can be done for static - environments like staging and production to unify the code path. You can save it - as an artifact with `echo $CI_ENVIRONMENT_URL > environment_url.txt` - in your job's `script`. -1. In the `performance` job, read the previous artifact into an environment - variable. In this case, use `$URL` because the sitespeed.io command - uses it for the URL parameter. Because Review App URLs are dynamic, define - the `URL` variable through `before_script` instead of `variables`. +1. In the `review` job: + 1. Generate a URL list file with the dynamic URL. + 1. Save the file as an artifact, for example with `echo $CI_ENVIRONMENT_URL > environment_url.txt` + in your job's `script`. + 1. Pass the list as the URL environment variable (which can be a URL or a file containing URLs) + to the `performance` job. 1. You can now run the sitespeed.io container against the desired hostname and paths. @@ -190,20 +174,21 @@ review: performance: dependencies: - review - before_script: - - export URL=$(cat environment_url.txt) + variables: + URL: environment_url.txt ``` -### Previous job definitions +### GitLab versions 12.3 and older -CAUTION: **Caution:** -Before GitLab 11.5, the Performance job and artifact had to be named specifically -to automatically extract report data and show it in the merge request widget. -While these old job definitions are still maintained, they have been deprecated -and may be removed in next major release, GitLab 12.0. -GitLab recommends you update your current `.gitlab-ci.yml` configuration to reflect that change. +Browser Performance Testing has gone through several changes since it's introduction. +In this section we'll detail these changes and how you can run the test based on your +GitLab version: -For GitLab 11.4 and earlier, the job should look like: +- In GitLab 12.4 [a job template was made available](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). +- In 13.2 the feature was renamed from `Performance` to `Browser Performance` with +additional template variables. The job name in the template is still `performance` +for compatibility reasons, but may be renamed to match in a future iteration. +- For 11.5 to 12.3 no template is available and the job has to be defined manually as follows: ```yaml performance: @@ -211,28 +196,45 @@ performance: image: docker:git variables: URL: https://example.com + SITESPEED_VERSION: 13.3.0 + SITESPEED_OPTIONS: '' services: - docker:stable-dind script: - mkdir gitlab-exporter - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js - mkdir sitespeed-results - - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:$SITESPEED_VERSION --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL $SITESPEED_OPTIONS - mv sitespeed-results/data/performance.json performance.json artifacts: paths: - performance.json - sitespeed-results/ + reports: + performance: performance.json ``` - +Upgrading to the latest version and using the templates is recommended, to ensure +you receive the latest updates, including updates to the sitespeed.io versions. diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 7b4d4b668d5..36acba032ff 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -1,8 +1,11 @@ --- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- -# Code Quality **(STARTER)** +# Code Quality > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. @@ -22,8 +25,13 @@ Code Quality: DevOps](../../../topics/autodevops/stages.md#auto-code-quality-starter). - Can be extended through [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) or a [custom tool](#implementing-a-custom-tool). +## Code Quality Widget + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in 13.2. + Going a step further, GitLab can show the Code Quality report right -in the merge request widget area: +in the merge request widget area if a report from the target branch is available to compare to: ![Code Quality Widget](img/code_quality.png) @@ -79,7 +87,7 @@ include: The above example will create a `code_quality` job in your CI/CD pipeline which will scan your source code for code quality issues. The report will be saved as a -[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter) +[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality) that you can later download and analyze. It's also possible to override the URL to the Code Quality image by @@ -237,7 +245,7 @@ do this: 1. Define a job in your `.gitlab-ci.yml` file that generates the [Code Quality report - artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter). + artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality). 1. Configure your tool to generate the Code Quality report artifact as a JSON file that implements a subset of the [Code Climate spec](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types). @@ -273,11 +281,11 @@ NOTE: **Note:** Although the Code Climate spec supports more properties, those are ignored by GitLab. -## Code Quality reports +## Code Quality reports **(STARTER)** Once the Code Quality job has completed: -- The full list of code quality violations generated by a pipeline is available in the +- The full list of code quality violations generated by a pipeline is shown in the Code Quality tab of the Pipeline Details page. - Potential changes to code quality are shown directly in the merge request. The Code Quality widget in the merge request compares the reports from the base and head of the branch, @@ -286,8 +294,43 @@ Once the Code Quality job has completed: [downloadable artifact](../../../ci/pipelines/job_artifacts.md#downloading-artifacts) for the `code_quality` job. +## Extending functionality + +### Using Analysis Plugins + +Should there be a need to extend the default functionality provided by Code Quality, as stated in [Code Quality](#code-quality), [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) are available. + +For example, to use the [SonarJava analyzer](https://docs.codeclimate.com/docs/sonar-java), +add a file named `.codeclimate.yml` containing the [enablement code](https://docs.codeclimate.com/docs/sonar-java#enable-the-plugin) +for the plugin to the root of your repository: + +```yaml +version: "2" +plugins: + sonar-java: + enabled: true +``` + +This adds SonarJava to the `plugins:` section of the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml) +included in your project. + +Changes to the `plugins:` section do not affect the `exclude_patterns` section of the +defeault `.codeclimate.yml`. See the Code Climate documentation for +[excluding files and folders](https://docs.codeclimate.com/docs/excluding-files-and-folders) +for more details. + +Here's [an example project](https://gitlab.com/jheimbuck_gl/jh_java_example_project) that uses Code Quality with a `.codeclimate.yml` file. + ## Troubleshooting +### Changing the default configuration has no effect + +A common issue is that the terms `Code Quality` (GitLab specific) and `Code Climate` +(Engine used by GitLab) are very similar. You must add a **`.codeclimate.yml`** file +to change the default configuration, **not** a `.codequality.yml` file. If you use +the wrong filename, the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml) +is still used. + ### No Code Quality report is displayed in a Merge Request This can be due to multiple reasons: @@ -295,6 +338,7 @@ This can be due to multiple reasons: - You just added the Code Quality job in your `.gitlab-ci.yml`. The report does not have anything to compare to yet, so no information can be displayed. Future merge requests will have something to compare to. +- Your pipeline is not set to run the code quality job on your default branch. If there is no report generated from the default branch, your MR branch reports will not have anything to compare to. - If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks), nothing will be displayed. - The [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) CI/CD diff --git a/doc/user/project/merge_requests/fail_fast_testing.md b/doc/user/project/merge_requests/fail_fast_testing.md new file mode 100644 index 00000000000..619a6d04577 --- /dev/null +++ b/doc/user/project/merge_requests/fail_fast_testing.md @@ -0,0 +1,87 @@ +--- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Fail Fast Testing **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198550) in GitLab 13.1. + +For applications that use RSpec for running tests, we've introduced the `Verify/Failfast` +[template to run subsets of your test suite](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Verify/FailFast.gitlab-ci.yml), +based on the changes in your merge request. + +The template uses the [test_file_finder (`tff`) gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder/) +that accepts a list of files as input, and returns a list of spec (test) files +that it believes to be relevant to the input files. + +`tff` is designed for Ruby on Rails projects, so the `Verify/FailFast` template is +configured to run when changes to Ruby files are detected. By default, it runs in +the [`.pre` stage](../../../ci/yaml/README.md#pre-and-post) of a GitLab CI/CD pipeline, +before all other stages. + +## Example use case + +Fail fast testing is useful when adding new functionality to a project and adding +new automated tests. + +Your project could have hundreds of thousands of tests that take a long time to complete. +You may be confident that a new test will pass, but you have to wait for all the tests +to complete to verify it. This could take an hour or more, even when using parallelization. + +Fail fast testing gives you a faster feedback loop from the pipeline. It lets you +know quickly that the new tests are passing and the new functionality did not break +other tests. + +## Requirements + +This template requires: + +- A project built in Rails that uses RSpec for testing. +- CI/CD configured to: + - Use a Docker image with Ruby available. + - Use [Pipelines for Merge Requests](../../../ci/merge_request_pipelines/index.md#configuring-pipelines-for-merge-requests) +- [Pipelines for Merged Results](../../../ci/merge_request_pipelines/pipelines_for_merged_results/index.md#enable-pipelines-for-merged-results) + enabled in the project settings. + +## Configure Fast RSpec Failure + +We'll use the following plain RSpec configuration as a starting point. It installs all the +project gems and executes `rspec`, on merge request pipelines only. + +```yaml +rspec-complete: + stage: test + rules: + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + script: + - bundle install + - bundle exec rspec +``` + +To run the most relevant specs first instead of the whole suite, [`include`](../../../ci/yaml/README.md#include) +the template by adding the following to your CI/CD configuration: + +```yaml +include: + - template: Verify/FailFast.gitlab-ci.yml +``` + +### Example test loads + +For illustrative purposes, let's say our Rails app spec suite consists of 100 specs per model for ten models. + +If no Ruby files are changed: + +- `rspec-rails-modified-paths-specs` will not run any tests. +- `rspec-complete` will run the full suite of 1000 tests. + +If one Ruby model is changed, for example `app/models/example.rb`, then `rspec-rails-modified-paths-specs` +will run the 100 tests for `example.rb`: + +- If all of these 100 tests pass, then the full `rspec-complete` suite of 1000 tests is allowed to run. +- If any of these 100 tests fail, they will fail quickly, and `rspec-complete` will not run any tests. + +The final case saves resources and time as the full 1000 test suite does not run. diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index 32eb0c73ed4..9ac0f3ee42e 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -57,7 +57,7 @@ request's page at the top-right side: - [Close issues automatically](#merge-requests-to-close-issues) when they are merged. - Enable the [delete source branch when merge request is accepted](#deleting-the-source-branch) option to keep your repository clean. - Enable the [squash commits when merge request is accepted](squash_and_merge.md) option to combine all the commits into one before merging, thus keep a clean commit history in your repository. -- Set the merge request as a [Work In Progress (WIP)](work_in_progress_merge_requests.md) to avoid accidental merges before it is ready. +- Set the merge request as a [**Draft**](work_in_progress_merge_requests.md) to avoid accidental merges before it is ready. Once you have created the merge request, you can also: diff --git a/doc/user/project/merge_requests/img/browser_performance_testing.png b/doc/user/project/merge_requests/img/browser_performance_testing.png index eea77fb8b93..c270462f7a8 100644 Binary files a/doc/user/project/merge_requests/img/browser_performance_testing.png and b/doc/user/project/merge_requests/img/browser_performance_testing.png differ diff --git a/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png b/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png new file mode 100644 index 00000000000..beb12c581d6 Binary files /dev/null and b/doc/user/project/merge_requests/img/draft_blocked_merge_button_v13_2.png differ diff --git a/doc/user/project/merge_requests/img/file_by_file_v13_2.png b/doc/user/project/merge_requests/img/file_by_file_v13_2.png new file mode 100644 index 00000000000..e3114ebabad Binary files /dev/null and b/doc/user/project/merge_requests/img/file_by_file_v13_2.png differ diff --git a/doc/user/project/merge_requests/img/load_performance_testing.png b/doc/user/project/merge_requests/img/load_performance_testing.png new file mode 100644 index 00000000000..3a58e9c28d4 Binary files /dev/null and b/doc/user/project/merge_requests/img/load_performance_testing.png differ diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png deleted file mode 100644 index 761690d1e0c..00000000000 Binary files a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png and /dev/null differ diff --git a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png deleted file mode 100644 index ab2c8425b83..00000000000 Binary files a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png and /dev/null differ diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 5d2813f5bfc..f68fc7d7b45 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -19,7 +19,7 @@ A. Consider you're a software developer working in a team: 1. You checkout a new branch, and submit your changes through a merge request 1. You gather feedback from your team -1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) **(STARTER)** +1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) 1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD 1. You avoid using dependencies whose license is not compatible with your project with [License Compliance reports](../../compliance/license_compliance/index.md) **(ULTIMATE)** 1. You request the [approval](merge_request_approvals.md) from your manager **(STARTER)** diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md new file mode 100644 index 00000000000..3239269109d --- /dev/null +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -0,0 +1,197 @@ +--- +stage: Verify +group: Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Load Performance Testing **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. + +With Load Performance Testing, you can test the impact of any pending code changes +to your application's backend in [GitLab CI/CD](../../../ci/README.md). + +GitLab uses [k6](https://k6.io/), a free and open source +tool, for measuring the system performance of applications under +load. + +Unlike [Browser Performance Testing](browser_performance_testing.md), which is +used to measure how web sites perform in client browsers, Load Performance Testing +can be used to perform various types of [load tests](https://k6.io/docs/#use-cases) +against application endpoints such as APIs, Web Controllers, and so on. +This can be used to test how the backend or the server performs at scale. + +For example, you can use Load Performance Testing to perform many concurrent +GET calls to a popular API endpoint in your application to see how it performs. + +## How Load Performance Testing works + +First, define a job in your `.gitlab-ci.yml` file that generates the +[Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +GitLab checks this report, compares key load performance metrics +between the source and target branches, and then shows the information in a merge request widget: + +![Load Performance Widget](img/load_performance_testing.png) + +Next, you need to configure the test environment and write the k6 test. + +The key performance metrics that the merge request widget shows after the test completes are: + +- Checks: The percentage pass rate of the [checks](https://k6.io/docs/using-k6/checks) configured in the k6 test. +- TTFB P90: The 90th percentile of how long it took to start receiving responses, aka the [Time to First Byte](https://en.wikipedia.org/wiki/Time_to_first_byte) (TTFB). +- TTFB P95: The 95th percentile for TTFB. +- RPS: The average requests per second (RPS) rate the test was able to achieve. + +NOTE: **Note:** +If the Load Performance report has no data to compare, such as when you add the +Load Performance job in your `.gitlab-ci.yml` for the very first time, +the Load Performance report widget won't show. It must have run at least +once on the target branch (`master`, for example), before it will display in a +merge request targeting that branch. + +## Configure the Load Performance Testing job + +Configuring your Load Performance Testing job can be broken down into several distinct parts: + +- Determine the test parameters such as throughput, and so on. +- Set up the target test environment for load performance testing. +- Design and write the k6 test. + +### Determine the test parameters + +The first thing you need to do is determine the [type of load test](https://k6.io/docs/test-types/introduction) +you want to run, and how it will run (for example, the number of users, throughput, and so on). + +Refer to the [k6 docs](https://k6.io/docs/), especially the [k6 testing guides](https://k6.io/docs/testing-guides), +for guidance on the above and more. + +### Test Environment setup + +A large part of the effort around load performance testing is to prepare the target test environment +for high loads. You should ensure it's able to handle the +[throughput](https://k6.io/blog/monthly-visits-concurrent-users) it will be tested with. + +It's also typically required to have representative test data in the target environment +for the load performance test to use. + +We strongly recommend [not running these tests against a production environment](https://k6.io/our-beliefs#load-test-in-a-pre-production-environment). + +### Write the load performance test + +After the environment is prepared, you can write the k6 test itself. k6 is a flexible +tool and can be used to run [many kinds of performance tests](https://k6.io/docs/test-types/introduction). +Refer to the [k6 documentation](https://k6.io/docs/) for detailed information on how to write tests. + +### Configure the test in GitLab CI/CD + +When your k6 test is ready, the next step is to configure the load performance +testing job in GitLab CI/CD. The easiest way to do this is to use the +[`Verify/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Load-Performance-Testing.gitlab-ci.yml) +template that is included with GitLab. + +NOTE: **Note:** +For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual +test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) +for spec details. The [default shared GitLab.com runners](../../gitlab_com/#linux-shared-runners) +likely have insufficient specs to handle most large k6 tests. + +This template runs the +[k6 Docker container](https://hub.docker.com/r/loadimpact/k6/) in the job and provides several ways to customize the +job. + +An example configuration workflow: + +1. Set up a GitLab Runner that can run Docker containers, such as a Runner using the + [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). +1. Configure the default Load Performance Testing CI job in your `.gitlab-ci.yml` file. + You need to include the template and configure it with variables: + + ```yaml + include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + + load_performance: + variables: + K6_TEST_FILE: + ``` + +The above example creates a `load_performance` job in your CI/CD pipeline that runs +the k6 test. + +NOTE: **Note:** +For Kubernetes setups a different template should be used: [`Jobs/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Load-Performance-Testing.gitlab-ci.yml). + +k6 has [various options](https://k6.io/docs/using-k6/options) to configure how it will run tests, such as what throughput (RPS) to run with, +how long the test should run, and so on. Almost all options can be configured in the test itself, but as +you can also pass command line options via the `K6_OPTIONS` variable. + +For example, you can override the duration of the test with a CLI option: + +```yaml + include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + + load_performance: + variables: + K6_TEST_FILE: + K6_OPTIONS: '--duration 30s' +``` + +GitLab only displays the key performance metrics in the MR widget if k6's results are saved +via [summary export](https://k6.io/docs/results-visualization/json#summary-export) +as a [Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +The latest Load Performance artifact available is always used. + +If [GitLab Pages](../pages/index.md) is enabled, you can view the report directly in your browser. + +### Load Performance testing in Review Apps + +The CI/CD YAML configuration example above works for testing against static environments, +but it can be extended to work with [review apps](../../../ci/review_apps) or +[dynamic environments](../../../ci/environments) with a few extra steps. + +The best approach is to capture the dynamic URL into a custom environment variable that +is then [inherited](../../../ci/variables/README.md#inherit-environment-variables) +by the `load_performance` job. The k6 test script to be run should then be configured to +use that environment URL, such as: ``http.get(`${__ENV.ENVIRONMENT_URL`})``. + +For example: + +1. In the `review` job: + 1. Capture the dynamic URL and save it into a `.env` file, e.g. `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. + 1. Set the `.env` file to be an [`artifacts:reports:dotenv` report](../../../ci/variables/README.md#inherit-environment-variables). +1. Set the `load_performance` job to depend on the review job, so it inherits the environment variable. +1. Configure the k6 test script to use the environment variable in it's steps. + +Your `.gitlab-ci.yml` file might be similar to: + +```yaml +stages: + - deploy + - performance + +include: + template: Verify/Load-Performance-Testing.gitlab-ci.yml + +review: + stage: deploy + environment: + name: review/$CI_COMMIT_REF_NAME + url: http://$CI_ENVIRONMENT_SLUG.example.com + script: + - run_deploy_script + - echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env + artifacts: + reports: + dotenv: + review.env + rules: + - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + +load_performance: + dependencies: + - review + rules: + - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. +``` diff --git a/doc/user/project/merge_requests/merge_request_dependencies.md b/doc/user/project/merge_requests/merge_request_dependencies.md index a1012e27d32..65f3cb1e0d5 100644 --- a/doc/user/project/merge_requests/merge_request_dependencies.md +++ b/doc/user/project/merge_requests/merge_request_dependencies.md @@ -38,15 +38,15 @@ the `awesome-project` merge request before the `awesome-lib` one would break the `master`branch. The `awesome-project` merge request could be [marked as -WIP](work_in_progress_merge_requests.md), -and the reason for the WIP stated included in the comments. However, this +**Draft**](work_in_progress_merge_requests.md), +and the reason for the draft stated included in the comments. However, this requires the state of the `awesome-lib` merge request to be manually tracked, and doesn't scale well if the `awesome-project` merge request depends on changes to **several** other projects. By making the `awesome-project` merge request depend on the `awesome-lib` merge request instead, this relationship is -automatically tracked by GitLab, and the WIP state can be used to +automatically tracked by GitLab, and the draft state can be used to communicate the readiness of the code in each individual merge request instead. diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index d45ccdc9be9..7d90c9f3cd7 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -1,36 +1,38 @@ --- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, concepts --- # Merge when pipeline succeeds -When reviewing a merge request that looks ready to merge but still has one or -more CI jobs running, you can set it to be merged automatically when the -jobs pipeline succeeds. This way, you don't have to wait for the jobs to +When reviewing a merge request that looks ready to merge but still has a +pipeline running, you can set it to merge automatically when the +pipeline succeeds. This way, you don't have to wait for the pipeline to finish and remember to merge the request manually. ![Enable](img/merge_when_pipeline_succeeds_enable.png) ## How it works -When you hit the "Merge When Pipeline Succeeds" button, the status of the merge -request will be updated to represent the impending merge. If you cannot wait -for the pipeline to succeed and want to merge immediately, this option is -available in the dropdown menu on the right of the main button. +When you click "Merge When Pipeline Succeeds", the status of the merge +request is updated to show the impending merge. If you can't wait +for the pipeline to succeed, you can choose **Merge immediately** +in the dropdown menu on the right of the main button. -Both team developers and the author of the merge request have the option to -cancel the automatic merge if they find a reason why it shouldn't be merged -after all. +The author of the merge request and project members with developer permissions can +cancel the automatic merge at any time before the pipeline finishes. ![Status](img/merge_when_pipeline_succeeds_status.png) -When the pipeline succeeds, the merge request will automatically be merged. +When the pipeline succeeds, the merge request is automatically merged. When the pipeline fails, the author gets a chance to retry any failed jobs, or to push new commits to fix the failure. When the jobs are retried and succeed on the second try, the merge request -will automatically be merged after all. When the merge request is updated with -new commits, the automatic merge is automatically canceled to allow the new +is automatically merged. When the merge request is updated with +new commits, the automatic merge is canceled to allow the new changes to be reviewed. ## Only allow merge requests to be merged if the pipeline succeeds @@ -42,7 +44,7 @@ or if there are threads to be resolved. This works for both: - Pipelines run from an [external CI integration](../integrations/overview.md#integrations-listing) As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md) -will not disable this feature, as it will still be possible to use pipelines from external +does not disable this feature, as it is possible to use pipelines from external CI providers with this feature. To enable it, you must: 1. Navigate to your project's **Settings > General** page. @@ -50,14 +52,40 @@ CI providers with this feature. To enable it, you must: 1. In the **Merge checks** subsection, select the **Pipelines must succeed** checkbox. 1. Press **Save** for the changes to take effect. -NOTE: **Note:** This setting also prevents merge requests from being merged if there is no pipeline. +This setting also prevents merge requests from being merged if there is no pipeline. -![Pipelines must succeed settings](img/merge_when_pipeline_succeeds_only_if_succeeds_settings.png) +### Limitations + +When this setting is enabled, a merge request is prevented from being merged if there +is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced) +or [`rules`](../../../ci/yaml/README.md#rules) are used and they don't generate any pipelines. + +You should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) +and that it's successful. + +If both a branch pipeline and a merge request pipeline are triggered for a single +merge request, only the success or failure of the *merge request pipeline* is checked. +If the merge request pipeline is configured with fewer jobs than the branch pipeline, +it could allow code that fails tests to be merged: + +```yaml +branch-pipeline-job: + rules: + - if: '$CI_PIPELINE_SOURCE == "push"' + script: + - echo "Code testing scripts here, for example." -From now on, every time the pipeline fails you will not be able to merge the -merge request from the UI, until you make all relevant jobs pass. +merge-request-pipeline-job: + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + script: + - echo "No tests run, but this pipeline always succeeds and enables merge." + - echo true +``` -![Only allow merge if pipeline succeeds message](img/merge_when_pipeline_succeeds_only_if_succeeds_msg.png) +You should avoid configuration like this, and only use branch (`push`) pipelines +or merge request pipelines, when possible. See [`rules` documentation](../../../ci/yaml/README.md#differences-between-rules-and-onlyexcept) +for details on avoiding two pipelines for a single merge request. ### Skipped pipelines @@ -72,20 +100,10 @@ merge requests from being merged. To change this behavior: 1. In the **Merge checks** subsection, select the **Skipped pipelines are considered successful** checkbox. 1. Press **Save** for the changes to take effect. -### Limitations - -When this setting is enabled, a merge request is prevented from being merged if there is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced) rules are used and they don't generate any pipelines. - -Users that expect to be able to merge a merge request in this scenario should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54226) and that it's successful. +## From the command line -For example, to that on merge requests there is always a passing job even though `only/except` rules may not generate any other jobs: - -```yaml -enable_merge: - only: [merge_requests] - script: - - echo true -``` +You can use [Push Options](../push_options.md) to enable merge when pipeline succeeds +for a merge request when pushing from the command line. - -## Use it from the command line - -You can use [Push Options](../push_options.md) to trigger this feature when -pushing. diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index a3ad41d8dd8..162ebdf157d 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -64,6 +64,43 @@ list. ![Merge request diff file navigation](img/merge_request_diff_file_navigation.png) +### File-by-file diff navigation + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222790) in GitLab 13.2. +> - It's deployed behind a feature flag, disabled by default. +> - It's enabled on GitLab.com. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-file-by-file-diff-navigation-core-only). + +For larger merge requests it might sometimes be useful to review single files at a time. To enable, +from your avatar on the top-right navbar, click **Settings**, and go to **Preferences** on the left +sidebar. Scroll down to the **Behavior** section and select **Show one file at a time on merge request's Changes tab**. +Click **Save changes** to apply. + +From there, when reviewing merge requests' **Changes** tab, you will see only one file at a time. You can then click the buttons **Prev** and **Next** to view the other files changed. + +![File-by-file diff navigation](img/file_by_file_v13_2.png) + +#### Enable or disable file-by-file diff navigation **(CORE ONLY)** + +File-by-file diff navigation is under development but ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:view_diffs_file_by_file) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:view_diffs_file_by_file>) +``` + ### Merge requests commit navigation > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18140) in GitLab 13.0. diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 924334055b9..79eec059293 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -85,6 +85,60 @@ it. This is because squashing is only available when accepting a merge request, so a merge request may need to be rebased before squashing, even though squashing can itself be considered equivalent to rebasing. +## Squash Commits Options + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17613) in GitLab 13.2. +> - It's deployed behind a feature flag, disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-squash-commit-options-core-only). **(CORE ONLY)** + +With Squash Commits Options you can configure the behavior of Squash and Merge for your project. +To set it up, navigate to your project's **Settings > General** and expand **Merge requests**. +You will find the following options to choose, which will affect existing and new merge requests +submitted to your project: + +- **Do not allow**: users cannot use Squash and Merge to squash all the commits immediately before + merging. The checkbox to enable or disable it will be unchecked and hidden from the users. +- **Allow**: users will have the option to enable Squash and Merge on a merge request basis. + The checkbox will be unchecked (disabled) by default, but and the user is allowed to enable it. +- **Encourage**: users will have the option to enable Squash and Merge on a merge request basis. + The checkbox will be checked (enabled) by default to encourage its use, but the user is allowed to + disable it. +- **Require**: Squash and Merge is enabled for all merge requests, so it will always be performed. + The checkbox to enable or disable it will be checked and hidden from the users. + +The Squash and Merge checkbox is displayed when you create a merge request and when you edit the description of an existing one, except when Squash Commit Options is set to **Do not allow** or **Require**. + +NOTE: **Note:** +If your project is set to **Do not allow** Squash and Merge, the users still have the option to +squash commits locally through the command line and force-push to their remote branch before merging. + +### Enable or disable Squash Commit Options **(CORE ONLY)** + +Squash Commit Options is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. Squash Commit Options can be enabled or disabled per-project. + +To enable it: + +```ruby +# Instance-wide +Feature.enable(:squash_options) +# or by project +Feature.enable(:squash_options, Project.find()) +``` + +To disable it: + +```ruby +# Instance-wide +Feature.disable(:squash_options) +# or by project +Feature.disable(:squash_options, Project.find()) +``` +